uber / cadence
72%
master: 72%

LAST BUILD BRANCH: fixit
DEFAULT BRANCH: master
Repo Added
Files 727
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH v1.2.11-prerelease10
branch: v1.2.11-prerelease10
CHANGE BRANCH
x

coverage: 71.507% (+0.07%) from 71.438%
01906f58-9152-4384-8df5-75027bc8a610

push

buildkite

web-flow 
Global ratelimiter: everything else (#6141)

After too many attempts to break this apart and build different portions in self-contained ways, and running into various inter-dependent roadblocks... I just gave up and did it all at once.

# Rollout plan for people who don't want or need this system

Do nothing :)

As of this PR, you'll use "disabled" and that should be as close to "no changes at all" as possible.
Soon, you'll get "local", and then you'll have some new metrics you can use (or ignore) but otherwise no behavior changes.

And that'll be it.  The "global" load-balanced stuff is likely to remain opt-in.

# Rollout plan for us

For deployment: any order is fine / should not behave (too) badly.  Even if "global" or either shadow mode is selected on the initial deploy.  Frontends will have background `RatelimitUpdate` request failures until History is deployed, but that'll just mean it continues to use the "local" internal fallback and that's in practice the same behavior as "local" or "disabled", just slightly noisier.

The _smoothest_ deployment is: deploy everything on "disabled" or "local" (the default(s), so no requests are sent until deploy is done), then switch to "local-shadow-global" to warm global limiters / check that it's working, then "global" to use the global behavior.  

Rolling back is just the opposite.  Ideally disable things first to stop the requests, but even if you don't it should be fine.

In more detail:

1. At merge time, this will set the "key mode" (`frontend.globalRatelimiterMode`) to "disabled", which gets as close as is reasonably possible to acting _exactly_ like it did before this PR.
   - This is also effectively the panic button for the initial rollout.
2. Once that proves to not immediately explode, switch to "local" for all keys.  This will keep the current ratelimiter rates, but will start collecting and emitting ratelimiter-usage metrics, so we can make sure that doesn't explode eithe... (continued)

688 of 850 new or added lines in 29 files covered. (80.94%)

15 existing lines in 7 files now uncovered.

105296 of 147252 relevant lines covered (71.51%)

2599.69 hits per line

Relevant lines Covered
147252 RELEVANT LINES 105296 COVERED LINES
2599.69 HITS PER LINE
Source Files on v1.2.11-prerelease10

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
01906f58... v1.2.11-prerelease10 Global ratelimiter: everything else (#6141) After too many attempts to break this apart and build different portions in self-contained ways, and running into various inter-dependent roadblocks... I just gave up and did it all at once. # Rollou... push web-flow buildkite
71.51
See All Builds (12090)