[{"data":1,"prerenderedAt":2850},["ShallowReactive",2],{"article-en-\u002Fbackend\u002Fapi-design\u002Frate-limiting-at-scale":3,"article-sibling-en-\u002Fbackend\u002Fapi-design\u002Frate-limiting-at-scale":1539,"surround-en-\u002Fbackend\u002Fapi-design\u002Frate-limiting-at-scale":2819,"related-en-\u002Fbackend\u002Fapi-design\u002Frate-limiting-at-scale":2826},{"id":4,"title":5,"body":6,"date":1519,"description":1520,"draft":1521,"extension":1522,"img":1523,"meta":1524,"navigation":145,"path":1525,"seo":1526,"slug":1527,"stem":1528,"tags":1529,"topics":1534,"__hash__":1538},"content\u002F2.backend\u002F2.api-design\u002F3.rate-limiting-at-scale.md","Why Redis Rate Limiting Breaks at Scale (and What Uber Does Instead)",{"type":7,"value":8,"toc":1509},"minimark",[9,13,21,32,37,45,421,428,432,435,438,660,673,896,899,903,910,913,945,960,967,971,982,985,1000,1003,1011,1018,1022,1029,1035,1042,1322,1333,1340,1344,1432,1435,1439,1442,1452,1462,1466,1502,1505],[10,11,12],"p",{},"Rate limiting on a single machine is a solved problem. You keep a counter, you refill it on a timer, you reject requests when it hits zero. Twenty lines of code and you are done.",[10,14,15,16,20],{},"The trouble starts the moment you have more than one machine. Now \"100 requests per second\" is a statement about the ",[17,18,19],"em",{},"whole fleet",", not about any one process. Each node only sees the slice of traffic that reached it, but the limit is global. Suddenly you need coordination, and coordination at scale is where most designs quietly fall apart.",[10,22,23,24,31],{},"This article walks the same path a real system walks: start with the in-memory token bucket, put it behind Redis so it works across nodes, watch that model break under load, and then look at what ",[25,26,30],"a",{"href":27,"rel":28},"https:\u002F\u002Fwww.uber.com\u002Fus\u002Fen\u002Fblog\u002Fubers-rate-limiting-system\u002F",[29],"nofollow","Uber built for their Global Rate Limiter"," instead. The goal is the reasoning, not a copy-paste library, so you can make the right call for your own services.",[33,34,36],"h2",{"id":35},"step-one-the-token-bucket-on-one-node","Step one: the token bucket on one node",[10,38,39,40,44],{},"The token bucket is the classic. A bucket holds up to ",[41,42,43],"code",{},"capacity"," tokens. Every request takes one token. Tokens refill at a steady rate. If the bucket is empty, the request is rejected. The capacity is what lets you absorb short bursts; the refill rate is your steady-state limit.",[46,47,52],"pre",{"className":48,"code":49,"language":50,"meta":51,"style":51},"language-go shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","type TokenBucket struct {\n    mu         sync.Mutex\n    tokens     float64\n    capacity   float64\n    refillRate float64 \u002F\u002F tokens per second\n    lastRefill time.Time\n}\n\nfunc (b *TokenBucket) Allow() bool {\n    b.mu.Lock()\n    defer b.mu.Unlock()\n\n    now := time.Now()\n    elapsed := now.Sub(b.lastRefill).Seconds()\n    b.tokens = math.Min(b.capacity, b.tokens+elapsed*b.refillRate)\n    b.lastRefill = now\n\n    if b.tokens >= 1 {\n        b.tokens--\n        return true\n    }\n    return false\n}\n","go","",[41,53,54,73,89,99,107,120,134,140,147,181,200,221,226,245,280,337,352,357,378,391,401,407,416],{"__ignoreMap":51},[55,56,59,63,67,70],"span",{"class":57,"line":58},"line",1,[55,60,62],{"class":61},"sMK4o","type",[55,64,66],{"class":65},"sBMFI"," TokenBucket",[55,68,69],{"class":61}," struct",[55,71,72],{"class":61}," {\n",[55,74,76,80,83,86],{"class":57,"line":75},2,[55,77,79],{"class":78},"sTEyZ","    mu         ",[55,81,82],{"class":65},"sync",[55,84,85],{"class":61},".",[55,87,88],{"class":65},"Mutex\n",[55,90,92,95],{"class":57,"line":91},3,[55,93,94],{"class":78},"    tokens     ",[55,96,98],{"class":97},"spNyl","float64\n",[55,100,102,105],{"class":57,"line":101},4,[55,103,104],{"class":78},"    capacity   ",[55,106,98],{"class":97},[55,108,110,113,116],{"class":57,"line":109},5,[55,111,112],{"class":78},"    refillRate ",[55,114,115],{"class":97},"float64",[55,117,119],{"class":118},"sHwdD"," \u002F\u002F tokens per second\n",[55,121,123,126,129,131],{"class":57,"line":122},6,[55,124,125],{"class":78},"    lastRefill ",[55,127,128],{"class":65},"time",[55,130,85],{"class":61},[55,132,133],{"class":65},"Time\n",[55,135,137],{"class":57,"line":136},7,[55,138,139],{"class":61},"}\n",[55,141,143],{"class":57,"line":142},8,[55,144,146],{"emptyLinePlaceholder":145},true,"\n",[55,148,150,153,156,160,163,166,169,173,176,179],{"class":57,"line":149},9,[55,151,152],{"class":61},"func",[55,154,155],{"class":61}," (",[55,157,159],{"class":158},"sHdIc","b ",[55,161,162],{"class":61},"*",[55,164,165],{"class":65},"TokenBucket",[55,167,168],{"class":61},")",[55,170,172],{"class":171},"s2Zo4"," Allow",[55,174,175],{"class":61},"()",[55,177,178],{"class":97}," bool",[55,180,72],{"class":61},[55,182,184,187,189,192,194,197],{"class":57,"line":183},10,[55,185,186],{"class":78},"    b",[55,188,85],{"class":61},[55,190,191],{"class":78},"mu",[55,193,85],{"class":61},[55,195,196],{"class":171},"Lock",[55,198,199],{"class":61},"()\n",[55,201,203,207,210,212,214,216,219],{"class":57,"line":202},11,[55,204,206],{"class":205},"s7zQu","    defer",[55,208,209],{"class":78}," b",[55,211,85],{"class":61},[55,213,191],{"class":78},[55,215,85],{"class":61},[55,217,218],{"class":171},"Unlock",[55,220,199],{"class":61},[55,222,224],{"class":57,"line":223},12,[55,225,146],{"emptyLinePlaceholder":145},[55,227,229,232,235,238,240,243],{"class":57,"line":228},13,[55,230,231],{"class":78},"    now ",[55,233,234],{"class":61},":=",[55,236,237],{"class":78}," time",[55,239,85],{"class":61},[55,241,242],{"class":171},"Now",[55,244,199],{"class":61},[55,246,248,251,253,256,258,261,264,267,269,272,275,278],{"class":57,"line":247},14,[55,249,250],{"class":78},"    elapsed ",[55,252,234],{"class":61},[55,254,255],{"class":78}," now",[55,257,85],{"class":61},[55,259,260],{"class":171},"Sub",[55,262,263],{"class":61},"(",[55,265,266],{"class":78},"b",[55,268,85],{"class":61},[55,270,271],{"class":78},"lastRefill",[55,273,274],{"class":61},").",[55,276,277],{"class":171},"Seconds",[55,279,199],{"class":61},[55,281,283,285,287,290,293,296,298,301,303,305,307,309,312,314,316,319,322,325,327,329,331,334],{"class":57,"line":282},15,[55,284,186],{"class":78},[55,286,85],{"class":61},[55,288,289],{"class":78},"tokens ",[55,291,292],{"class":61},"=",[55,294,295],{"class":78}," math",[55,297,85],{"class":61},[55,299,300],{"class":171},"Min",[55,302,263],{"class":61},[55,304,266],{"class":78},[55,306,85],{"class":61},[55,308,43],{"class":78},[55,310,311],{"class":61},",",[55,313,209],{"class":78},[55,315,85],{"class":61},[55,317,318],{"class":78},"tokens",[55,320,321],{"class":61},"+",[55,323,324],{"class":78},"elapsed",[55,326,162],{"class":61},[55,328,266],{"class":78},[55,330,85],{"class":61},[55,332,333],{"class":78},"refillRate",[55,335,336],{"class":61},")\n",[55,338,340,342,344,347,349],{"class":57,"line":339},16,[55,341,186],{"class":78},[55,343,85],{"class":61},[55,345,346],{"class":78},"lastRefill ",[55,348,292],{"class":61},[55,350,351],{"class":78}," now\n",[55,353,355],{"class":57,"line":354},17,[55,356,146],{"emptyLinePlaceholder":145},[55,358,360,363,365,367,369,372,376],{"class":57,"line":359},18,[55,361,362],{"class":205},"    if",[55,364,209],{"class":78},[55,366,85],{"class":61},[55,368,289],{"class":78},[55,370,371],{"class":61},">=",[55,373,375],{"class":374},"sbssI"," 1",[55,377,72],{"class":61},[55,379,381,384,386,388],{"class":57,"line":380},19,[55,382,383],{"class":78},"        b",[55,385,85],{"class":61},[55,387,318],{"class":78},[55,389,390],{"class":61},"--\n",[55,392,394,397],{"class":57,"line":393},20,[55,395,396],{"class":205},"        return",[55,398,400],{"class":399},"sfNiH"," true\n",[55,402,404],{"class":57,"line":403},21,[55,405,406],{"class":61},"    }\n",[55,408,410,413],{"class":57,"line":409},22,[55,411,412],{"class":205},"    return",[55,414,415],{"class":399}," false\n",[55,417,419],{"class":57,"line":418},23,[55,420,139],{"class":61},[10,422,423,424,427],{},"This is correct, fast, and lock-local. No network, no dependency, decisions in nanoseconds. It has exactly one flaw: the counter lives in this process's memory. Run ten replicas behind a load balancer and you have ten independent buckets, each enforcing the full limit. Your real global limit is now ",[41,425,426],{},"10 × capacity",". Nobody asked for that.",[33,429,431],{"id":430},"step-two-move-the-state-to-redis","Step two: move the state to Redis",[10,433,434],{},"The obvious fix is to pull the counter out of the process and into a store every node can see. Redis is the default choice: it is fast, atomic, and everyone already runs it.",[10,436,437],{},"The naive version is a counter per window with a TTL:",[46,439,441],{"className":48,"code":440,"language":50,"meta":51,"style":51},"func Allow(ctx context.Context, rdb *redis.Client, key string, limit int, window time.Duration) (bool, error) {\n    \u002F\u002F key like \"rl:user:42:1720512000\", bucketed by fixed window\n    count, err := rdb.Incr(ctx, key).Result()\n    if err != nil {\n        return false, err\n    }\n    if count == 1 {\n        rdb.Expire(ctx, key, window)\n    }\n    return count \u003C= int64(limit), nil\n}\n",[41,442,443,522,527,561,575,587,591,605,629,633,656],{"__ignoreMap":51},[55,444,445,447,449,451,454,457,459,462,464,467,470,473,475,478,480,483,486,488,491,494,496,499,501,503,506,508,510,513,515,518,520],{"class":57,"line":58},[55,446,152],{"class":61},[55,448,172],{"class":171},[55,450,263],{"class":61},[55,452,453],{"class":158},"ctx",[55,455,456],{"class":65}," context",[55,458,85],{"class":61},[55,460,461],{"class":65},"Context",[55,463,311],{"class":61},[55,465,466],{"class":158}," rdb",[55,468,469],{"class":61}," *",[55,471,472],{"class":65},"redis",[55,474,85],{"class":61},[55,476,477],{"class":65},"Client",[55,479,311],{"class":61},[55,481,482],{"class":158}," key",[55,484,485],{"class":97}," string",[55,487,311],{"class":61},[55,489,490],{"class":158}," limit",[55,492,493],{"class":97}," int",[55,495,311],{"class":61},[55,497,498],{"class":158}," window",[55,500,237],{"class":65},[55,502,85],{"class":61},[55,504,505],{"class":65},"Duration",[55,507,168],{"class":61},[55,509,155],{"class":61},[55,511,512],{"class":97},"bool",[55,514,311],{"class":61},[55,516,517],{"class":97}," error",[55,519,168],{"class":61},[55,521,72],{"class":61},[55,523,524],{"class":57,"line":75},[55,525,526],{"class":118},"    \u002F\u002F key like \"rl:user:42:1720512000\", bucketed by fixed window\n",[55,528,529,532,534,537,539,541,543,546,548,550,552,554,556,559],{"class":57,"line":91},[55,530,531],{"class":78},"    count",[55,533,311],{"class":61},[55,535,536],{"class":78}," err ",[55,538,234],{"class":61},[55,540,466],{"class":78},[55,542,85],{"class":61},[55,544,545],{"class":171},"Incr",[55,547,263],{"class":61},[55,549,453],{"class":78},[55,551,311],{"class":61},[55,553,482],{"class":78},[55,555,274],{"class":61},[55,557,558],{"class":171},"Result",[55,560,199],{"class":61},[55,562,563,565,567,570,573],{"class":57,"line":101},[55,564,362],{"class":205},[55,566,536],{"class":78},[55,568,569],{"class":61},"!=",[55,571,572],{"class":61}," nil",[55,574,72],{"class":61},[55,576,577,579,582,584],{"class":57,"line":109},[55,578,396],{"class":205},[55,580,581],{"class":399}," false",[55,583,311],{"class":61},[55,585,586],{"class":78}," err\n",[55,588,589],{"class":57,"line":122},[55,590,406],{"class":61},[55,592,593,595,598,601,603],{"class":57,"line":136},[55,594,362],{"class":205},[55,596,597],{"class":78}," count ",[55,599,600],{"class":61},"==",[55,602,375],{"class":374},[55,604,72],{"class":61},[55,606,607,610,612,615,617,619,621,623,625,627],{"class":57,"line":142},[55,608,609],{"class":78},"        rdb",[55,611,85],{"class":61},[55,613,614],{"class":171},"Expire",[55,616,263],{"class":61},[55,618,453],{"class":78},[55,620,311],{"class":61},[55,622,482],{"class":78},[55,624,311],{"class":61},[55,626,498],{"class":78},[55,628,336],{"class":61},[55,630,631],{"class":57,"line":149},[55,632,406],{"class":61},[55,634,635,637,639,642,645,647,650,653],{"class":57,"line":183},[55,636,412],{"class":205},[55,638,597],{"class":78},[55,640,641],{"class":61},"\u003C=",[55,643,644],{"class":97}," int64",[55,646,263],{"class":61},[55,648,649],{"class":78},"limit",[55,651,652],{"class":61},"),",[55,654,655],{"class":61}," nil\n",[55,657,658],{"class":57,"line":202},[55,659,139],{"class":61},[10,661,662,663,665,666,668,669,672],{},"Fixed windows have a well-known edge: a caller can send ",[41,664,649],{}," requests at the end of one window and ",[41,667,649],{}," more at the start of the next, so ",[41,670,671],{},"2 × limit"," in a hair over a second. The usual answer is a sliding window in a Lua script, run atomically inside Redis:",[46,674,678],{"className":675,"code":676,"language":677,"meta":51,"style":51},"language-lua shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","-- KEYS[1] = bucket key, ARGV = now (ms), window (ms), limit\nlocal now    = tonumber(ARGV[1])\nlocal window = tonumber(ARGV[2])\nlocal limit  = tonumber(ARGV[3])\n\nredis.call('ZREMRANGEBYSCORE', KEYS[1], 0, now - window)\nlocal count = redis.call('ZCARD', KEYS[1])\nif count \u003C limit then\n  redis.call('ZADD', KEYS[1], now, now)\n  redis.call('PEXPIRE', KEYS[1], window)\n  return 1\nend\nreturn 0\n","lua",[41,679,680,685,707,725,743,747,786,814,830,853,875,883,888],{"__ignoreMap":51},[55,681,682],{"class":57,"line":58},[55,683,684],{"class":118},"-- KEYS[1] = bucket key, ARGV = now (ms), window (ms), limit\n",[55,686,687,690,693,695,698,701,704],{"class":57,"line":75},[55,688,689],{"class":61},"local",[55,691,692],{"class":78}," now    ",[55,694,292],{"class":61},[55,696,697],{"class":171}," tonumber",[55,699,700],{"class":78},"(ARGV[",[55,702,703],{"class":374},"1",[55,705,706],{"class":78},"])\n",[55,708,709,711,714,716,718,720,723],{"class":57,"line":91},[55,710,689],{"class":61},[55,712,713],{"class":78}," window ",[55,715,292],{"class":61},[55,717,697],{"class":171},[55,719,700],{"class":78},[55,721,722],{"class":374},"2",[55,724,706],{"class":78},[55,726,727,729,732,734,736,738,741],{"class":57,"line":101},[55,728,689],{"class":61},[55,730,731],{"class":78}," limit  ",[55,733,292],{"class":61},[55,735,697],{"class":171},[55,737,700],{"class":78},[55,739,740],{"class":374},"3",[55,742,706],{"class":78},[55,744,745],{"class":57,"line":109},[55,746,146],{"emptyLinePlaceholder":145},[55,748,749,752,755,757,760,764,766,769,771,774,777,780,783],{"class":57,"line":122},[55,750,751],{"class":78},"redis.",[55,753,754],{"class":171},"call",[55,756,263],{"class":78},[55,758,759],{"class":61},"'",[55,761,763],{"class":762},"sfazB","ZREMRANGEBYSCORE",[55,765,759],{"class":61},[55,767,768],{"class":78},", KEYS[",[55,770,703],{"class":374},[55,772,773],{"class":78},"], ",[55,775,776],{"class":374},"0",[55,778,779],{"class":78},", now ",[55,781,782],{"class":61},"-",[55,784,785],{"class":78}," window)\n",[55,787,788,790,792,794,797,799,801,803,806,808,810,812],{"class":57,"line":136},[55,789,689],{"class":61},[55,791,597],{"class":78},[55,793,292],{"class":61},[55,795,796],{"class":78}," redis.",[55,798,754],{"class":171},[55,800,263],{"class":78},[55,802,759],{"class":61},[55,804,805],{"class":762},"ZCARD",[55,807,759],{"class":61},[55,809,768],{"class":78},[55,811,703],{"class":374},[55,813,706],{"class":78},[55,815,816,819,821,824,827],{"class":57,"line":142},[55,817,818],{"class":205},"if",[55,820,597],{"class":78},[55,822,823],{"class":61},"\u003C",[55,825,826],{"class":78}," limit ",[55,828,829],{"class":205},"then\n",[55,831,832,835,837,839,841,844,846,848,850],{"class":57,"line":149},[55,833,834],{"class":78},"  redis.",[55,836,754],{"class":171},[55,838,263],{"class":78},[55,840,759],{"class":61},[55,842,843],{"class":762},"ZADD",[55,845,759],{"class":61},[55,847,768],{"class":78},[55,849,703],{"class":374},[55,851,852],{"class":78},"], now, now)\n",[55,854,855,857,859,861,863,866,868,870,872],{"class":57,"line":183},[55,856,834],{"class":78},[55,858,754],{"class":171},[55,860,263],{"class":78},[55,862,759],{"class":61},[55,864,865],{"class":762},"PEXPIRE",[55,867,759],{"class":61},[55,869,768],{"class":78},[55,871,703],{"class":374},[55,873,874],{"class":78},"], window)\n",[55,876,877,880],{"class":57,"line":202},[55,878,879],{"class":205},"  return",[55,881,882],{"class":374}," 1\n",[55,884,885],{"class":57,"line":223},[55,886,887],{"class":205},"end\n",[55,889,890,893],{"class":57,"line":228},[55,891,892],{"class":205},"return",[55,894,895],{"class":374}," 0\n",[10,897,898],{},"This is accurate. Every node sees the same state, the limit is truly global, and the sliding window closes the boundary gap. For a lot of systems, this is the right answer and you can stop here.",[33,900,902],{"id":901},"step-three-where-the-redis-model-breaks","Step three: where the Redis model breaks",[10,904,905,906],{},"The Redis design has one property that looks harmless and becomes fatal: ",[907,908,909],"strong",{},"every single request does a network round trip before you know whether to allow it.",[10,911,912],{},"Walk through what that costs as you grow.",[914,915,916,927,933,939],"ul",{},[917,918,919,922,923,926],"li",{},[907,920,921],{},"Latency in the hot path."," Every request now waits on Redis before it proceeds. One round trip is a millisecond or two on a good day. You just added that to the p50 of ",[17,924,925],{},"every"," endpoint, and the tail is worse when Redis is busy.",[917,928,929,932],{},[907,930,931],{},"A shared dependency in the critical path."," If Redis is slow, every rate-limited service is slow. If Redis is down, you either fail open (no limiting) or fail closed (reject everything). You have coupled the availability of every service to one store.",[917,934,935,938],{},[907,936,937],{},"Hot keys."," A popular caller or a single hot endpoint funnels all its traffic to one key, which means one Redis shard. You cannot shard your way out of a single hot key; sharding splits keys, not the load on one key.",[917,940,941,944],{},[907,942,943],{},"The global-state tax."," Keeping an accurate, real-time global counter means every node talks to the same authoritative state constantly. Uber's team found that at their volume this was simply not viable, estimating \"hundreds of Redis clusters would be required to maintain accurate global state in real time.\"",[10,946,947,948,951,952,955,956,959],{},"The numbers that break this are not subtle. Uber runs on the order of ",[907,949,950],{},"80 million requests per second",", across ",[907,953,954],{},"1,100+ services"," and hundreds of thousands of hosts. At that scale, one round trip per request to a shared store is not a tax you can afford. The store ",[17,957,958],{},"is"," the bottleneck.",[10,961,962,963,966],{},"The root problem is the requirement itself: ",[907,964,965],{},"perfectly accurate global state, synchronously, on every request."," If you insist on that, you are stuck paying for it. So question the requirement.",[33,968,970],{"id":969},"the-shift-enforce-locally-coordinate-globally","The shift: enforce locally, coordinate globally",[10,972,973,974,977,978,981],{},"Here is the insight that unlocks scale. You do not need the exact global count on every request. You need each node to make a ",[17,975,976],{},"good"," local decision, using a global picture that is ",[17,979,980],{},"slightly stale",". A rate limit is a safety valve, not a bank ledger. If enforcement lags real traffic by a second or two, almost nothing breaks.",[10,983,984],{},"So split the two jobs that Redis was doing at once:",[986,987,988,994],"ol",{},[917,989,990,993],{},[907,991,992],{},"Enforcement"," happens locally, in the hot path, with zero network calls. Fast.",[917,995,996,999],{},[907,997,998],{},"Coordination"," happens out of band, asynchronously. Nodes report what they are seeing; a control plane aggregates it and tells them how hard to push back.",[10,1001,1002],{},"Uber's Global Rate Limiter is a three-tier feedback loop built on exactly this split:",[46,1004,1009],{"className":1005,"code":1007,"language":1008},[1006],"language-text","                                                  ┌───────────────────────┐\n                                                  │  Global \u002F Regional    │  aggregates zone usage,\n                                                  │  Controller           │  computes drop ratios,\n                                                  └───────────▲───────────┘  pushes directives down\n                                                      usage   │   drop ratio\n                                                              │\n                                                  ┌───────────┴───────────┐\n                                                  │   Zone Aggregators    │  sum per-host counts\n                                                  └───────────▲───────────┘  into zone usage\n                                                      counts  │   drop ratio\n                                                              │\n                                      ┌───────────────┬───────┴───────┬───────────────┐\n                                      │ Data plane    │ Data plane    │ Data plane    │  enforce locally,\n                                      │ (mesh proxy)  │ (mesh proxy)  │ (mesh proxy)  │  report counts up\n                                      └───────────────┴───────────────┴───────────────┘\n                                          ▲ decision in the hot path, no network hop\n","text",[41,1010,1007],{"__ignoreMap":51},[10,1012,1013,1014,1017],{},"The proxies decide every request locally. In the background they report per-host counts up to zone aggregators, which roll up to controllers, which compute how overloaded each bucket is and push a single number back down: the ",[907,1015,1016],{},"drop ratio",". The whole loop closes in a couple of seconds.",[33,1019,1021],{"id":1020},"the-algorithm-probabilistic-dropping","The algorithm: probabilistic dropping",[10,1023,1024,1025,1028],{},"Once a node has a drop ratio from the control plane, enforcement becomes almost trivial, and this is the part worth internalizing. Instead of counting toward a hard limit, each node drops a ",[17,1026,1027],{},"percentage"," of requests. The percentage is computed from how far over the limit the fleet is running:",[46,1030,1033],{"className":1031,"code":1032,"language":1008},[1006],"dropRatio = (actualRPS - limitRPS) \u002F actualRPS\n",[41,1034,1032],{"__ignoreMap":51},[10,1036,1037,1038,1041],{},"If the fleet is doing 1.5× its limit, that is ",[41,1039,1040],{},"(150 - 100) \u002F 150 ≈ 0.33",", so every node drops about a third of its requests. Add up the survivors across all nodes and you land back near the limit, without any node ever counting global state.",[46,1043,1045],{"className":48,"code":1044,"language":50,"meta":51,"style":51},"type Limiter struct {\n    dropRatio atomic.Value \u002F\u002F float64, updated by the control-plane loop\n}\n\n\u002F\u002F Allow is called in the hot path. No network, no lock on the fast path.\nfunc (l *Limiter) Allow() bool {\n    ratio, _ := l.dropRatio.Load().(float64)\n    if ratio \u003C= 0 {\n        return true \u002F\u002F fleet is under its limit, let everything through\n    }\n    \u002F\u002F drop each request independently with probability = ratio\n    return rand.Float64() >= ratio\n}\n\n\u002F\u002F updated asynchronously, out of the request path, every ~second\nfunc (l *Limiter) SetDropRatio(actualRPS, limitRPS float64) {\n    ratio := 0.0\n    if actualRPS > limitRPS {\n        ratio = (actualRPS - limitRPS) \u002F actualRPS\n    }\n    l.dropRatio.Store(ratio)\n}\n",[41,1046,1047,1058,1074,1078,1082,1087,1111,1143,1157,1167,1171,1176,1196,1200,1204,1209,1243,1253,1269,1293,1297,1318],{"__ignoreMap":51},[55,1048,1049,1051,1054,1056],{"class":57,"line":58},[55,1050,62],{"class":61},[55,1052,1053],{"class":65}," Limiter",[55,1055,69],{"class":61},[55,1057,72],{"class":61},[55,1059,1060,1063,1066,1068,1071],{"class":57,"line":75},[55,1061,1062],{"class":78},"    dropRatio ",[55,1064,1065],{"class":65},"atomic",[55,1067,85],{"class":61},[55,1069,1070],{"class":65},"Value",[55,1072,1073],{"class":118}," \u002F\u002F float64, updated by the control-plane loop\n",[55,1075,1076],{"class":57,"line":91},[55,1077,139],{"class":61},[55,1079,1080],{"class":57,"line":101},[55,1081,146],{"emptyLinePlaceholder":145},[55,1083,1084],{"class":57,"line":109},[55,1085,1086],{"class":118},"\u002F\u002F Allow is called in the hot path. No network, no lock on the fast path.\n",[55,1088,1089,1091,1093,1096,1098,1101,1103,1105,1107,1109],{"class":57,"line":122},[55,1090,152],{"class":61},[55,1092,155],{"class":61},[55,1094,1095],{"class":158},"l ",[55,1097,162],{"class":61},[55,1099,1100],{"class":65},"Limiter",[55,1102,168],{"class":61},[55,1104,172],{"class":171},[55,1106,175],{"class":61},[55,1108,178],{"class":97},[55,1110,72],{"class":61},[55,1112,1113,1116,1118,1121,1123,1126,1128,1131,1133,1136,1139,1141],{"class":57,"line":136},[55,1114,1115],{"class":78},"    ratio",[55,1117,311],{"class":61},[55,1119,1120],{"class":78}," _ ",[55,1122,234],{"class":61},[55,1124,1125],{"class":78}," l",[55,1127,85],{"class":61},[55,1129,1130],{"class":78},"dropRatio",[55,1132,85],{"class":61},[55,1134,1135],{"class":171},"Load",[55,1137,1138],{"class":61},"().(",[55,1140,115],{"class":97},[55,1142,336],{"class":61},[55,1144,1145,1147,1150,1152,1155],{"class":57,"line":142},[55,1146,362],{"class":205},[55,1148,1149],{"class":78}," ratio ",[55,1151,641],{"class":61},[55,1153,1154],{"class":374}," 0",[55,1156,72],{"class":61},[55,1158,1159,1161,1164],{"class":57,"line":149},[55,1160,396],{"class":205},[55,1162,1163],{"class":399}," true",[55,1165,1166],{"class":118}," \u002F\u002F fleet is under its limit, let everything through\n",[55,1168,1169],{"class":57,"line":183},[55,1170,406],{"class":61},[55,1172,1173],{"class":57,"line":202},[55,1174,1175],{"class":118},"    \u002F\u002F drop each request independently with probability = ratio\n",[55,1177,1178,1180,1183,1185,1188,1190,1193],{"class":57,"line":223},[55,1179,412],{"class":205},[55,1181,1182],{"class":78}," rand",[55,1184,85],{"class":61},[55,1186,1187],{"class":171},"Float64",[55,1189,175],{"class":61},[55,1191,1192],{"class":61}," >=",[55,1194,1195],{"class":78}," ratio\n",[55,1197,1198],{"class":57,"line":228},[55,1199,139],{"class":61},[55,1201,1202],{"class":57,"line":247},[55,1203,146],{"emptyLinePlaceholder":145},[55,1205,1206],{"class":57,"line":282},[55,1207,1208],{"class":118},"\u002F\u002F updated asynchronously, out of the request path, every ~second\n",[55,1210,1211,1213,1215,1217,1219,1221,1223,1226,1228,1231,1233,1236,1239,1241],{"class":57,"line":339},[55,1212,152],{"class":61},[55,1214,155],{"class":61},[55,1216,1095],{"class":158},[55,1218,162],{"class":61},[55,1220,1100],{"class":65},[55,1222,168],{"class":61},[55,1224,1225],{"class":171}," SetDropRatio",[55,1227,263],{"class":61},[55,1229,1230],{"class":158},"actualRPS",[55,1232,311],{"class":61},[55,1234,1235],{"class":158}," limitRPS",[55,1237,1238],{"class":97}," float64",[55,1240,168],{"class":61},[55,1242,72],{"class":61},[55,1244,1245,1248,1250],{"class":57,"line":354},[55,1246,1247],{"class":78},"    ratio ",[55,1249,234],{"class":61},[55,1251,1252],{"class":374}," 0.0\n",[55,1254,1255,1257,1260,1263,1266],{"class":57,"line":359},[55,1256,362],{"class":205},[55,1258,1259],{"class":78}," actualRPS ",[55,1261,1262],{"class":61},">",[55,1264,1265],{"class":78}," limitRPS ",[55,1267,1268],{"class":61},"{\n",[55,1270,1271,1274,1276,1278,1281,1283,1285,1287,1290],{"class":57,"line":380},[55,1272,1273],{"class":78},"        ratio ",[55,1275,292],{"class":61},[55,1277,155],{"class":61},[55,1279,1280],{"class":78},"actualRPS ",[55,1282,782],{"class":61},[55,1284,1235],{"class":78},[55,1286,168],{"class":61},[55,1288,1289],{"class":61}," \u002F",[55,1291,1292],{"class":78}," actualRPS\n",[55,1294,1295],{"class":57,"line":393},[55,1296,406],{"class":61},[55,1298,1299,1302,1304,1306,1308,1311,1313,1316],{"class":57,"line":403},[55,1300,1301],{"class":78},"    l",[55,1303,85],{"class":61},[55,1305,1130],{"class":78},[55,1307,85],{"class":61},[55,1309,1310],{"class":171},"Store",[55,1312,263],{"class":61},[55,1314,1315],{"class":78},"ratio",[55,1317,336],{"class":61},[55,1319,1320],{"class":57,"line":409},[55,1321,139],{"class":61},[10,1323,1324,1325,1328,1329,1332],{},"Look at what this bought us. ",[41,1326,1327],{},"Allow()"," is a single atomic load and a random number: no lock contention, no Redis, no hot key. The expensive part, computing the ratio, moved out of the request path entirely and runs once a second on aggregated data. Uber's move to this model cut tail latencies dramatically, with p99.5 dropping by up to ",[907,1330,1331],{},"90%"," once the Redis round trip was gone.",[10,1334,1335,1336,1339],{},"The catch is honest and worth stating: because the drop ratio rides on data aggregated every second, enforcement can lag real traffic by ",[907,1337,1338],{},"2 to 3 seconds",". For a sudden, extremely brief spike, the system reacts a beat late. In practice that window rarely matters, and trading it away is what makes the whole thing scale.",[33,1341,1343],{"id":1342},"the-tradeoffs-side-by-side","The tradeoffs, side by side",[1345,1346,1347,1362],"table",{},[1348,1349,1350],"thead",{},[1351,1352,1353,1356,1359],"tr",{},[1354,1355],"th",{},[1354,1357,1358],{},"Centralized Redis",[1354,1360,1361],{},"Local enforce + global coordinate",[1363,1364,1365,1377,1388,1399,1410,1421],"tbody",{},[1351,1366,1367,1371,1374],{},[1368,1369,1370],"td",{},"Decision path",[1368,1372,1373],{},"Network round trip per request",[1368,1375,1376],{},"Local, in-memory",[1351,1378,1379,1382,1385],{},[1368,1380,1381],{},"Accuracy",[1368,1383,1384],{},"Exact, real-time",[1368,1386,1387],{},"Approximate, ~1 to 3s stale",[1351,1389,1390,1393,1396],{},[1368,1391,1392],{},"Failure mode",[1368,1394,1395],{},"Redis is a shared SPOF",[1368,1397,1398],{},"Fails open per node, no shared dependency",[1351,1400,1401,1404,1407],{},[1368,1402,1403],{},"Hot keys",[1368,1405,1406],{},"Funnel to one shard",[1368,1408,1409],{},"None, no shared counter",[1351,1411,1412,1415,1418],{},[1368,1413,1414],{},"Latency added",[1368,1416,1417],{},"1 to 2ms+ per request",[1368,1419,1420],{},"Effectively zero",[1351,1422,1423,1426,1429],{},[1368,1424,1425],{},"Scales to",[1368,1427,1428],{},"Thousands of RPS comfortably",[1368,1430,1431],{},"Tens of millions of RPS",[10,1433,1434],{},"There is no free lunch here, only a swap: you give up exact real-time accuracy and gain the ability to enforce limits at a scale where the accurate model physically cannot run.",[33,1436,1438],{"id":1437},"two-details-that-make-it-production-grade","Two details that make it production-grade",[10,1440,1441],{},"The core algorithm is the headline, but two operational pieces are what let a team actually trust it.",[10,1443,1444,1447,1448,1451],{},[907,1445,1446],{},"Fail open."," If the control plane goes quiet, nodes keep serving traffic with whatever ratio they last held, or none. A rate limiter that takes the whole system down when ",[17,1449,1450],{},"it"," fails is worse than no rate limiter. Enforcement being local is what makes this safe: a node needs nothing external to keep answering requests.",[10,1453,1454,1457,1458,1461],{},[907,1455,1456],{},"Shadow mode."," Before a new limit enforces anything, run it in shadow: compute what ",[17,1459,1460],{},"would"," have been dropped and emit it as a metric, drop nothing. Teams watch the graph, confirm the limit is sane against real traffic, and only then flip to enforce. Uber pairs this with automated tuning that derives limits from weeks of observed peaks plus headroom, so limits track traffic instead of rotting in a YAML file someone set two years ago.",[33,1463,1465],{"id":1464},"takeaways","Takeaways",[914,1467,1468,1474,1480,1486,1496],{},[917,1469,1470,1473],{},[907,1471,1472],{},"Single-node rate limiting is easy; the whole problem is coordination."," Every design decision is really a decision about how much accuracy in that coordination you are willing to trade for scale.",[917,1475,1476,1479],{},[907,1477,1478],{},"A shared store on the hot path is a scaling ceiling."," It works beautifully until the round trip per request, and the store itself, becomes the bottleneck. Know where that ceiling is for your traffic.",[917,1481,1482,1485],{},[907,1483,1484],{},"Relaxing \"exact and real-time\" to \"approximate and slightly stale\" changes the whole design."," A 1 to 3 second lag lets you move enforcement local and delete the shared dependency entirely.",[917,1487,1488,1491,1492,1495],{},[907,1489,1490],{},"Probabilistic dropping turns a hard global count into a trivial local decision:"," drop with probability ",[41,1493,1494],{},"(actual − limit) \u002F actual",", computed out of band.",[917,1497,1498,1501],{},[907,1499,1500],{},"Infrastructure-level limiting beats application-level at the top of the scale curve,"," because it enforces uniformly across every service without each one reimplementing the same counter.",[10,1503,1504],{},"If you are running thousands of requests per second, the Redis sliding window is probably the right tool and you should not overbuild. But if you are staring at the point where a round trip per request stops being affordable, the answer is not a bigger Redis. It is to stop asking for a globally accurate count on every request, and let each node decide for itself with a number that is good enough.",[1506,1507,1508],"style",{},"html pre.shiki code .sMK4o, html code.shiki .sMK4o{--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF}html pre.shiki code .sBMFI, html code.shiki .sBMFI{--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B}html pre.shiki code .sTEyZ, html code.shiki .sTEyZ{--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8}html pre.shiki code .spNyl, html code.shiki .spNyl{--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA}html pre.shiki code .sHwdD, html code.shiki .sHwdD{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic}html pre.shiki code .sHdIc, html code.shiki .sHdIc{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#EEFFFF;--shiki-default-font-style:italic;--shiki-dark:#BABED8;--shiki-dark-font-style:italic}html pre.shiki code .s2Zo4, html code.shiki .s2Zo4{--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF}html pre.shiki code .s7zQu, html code.shiki .s7zQu{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic}html pre.shiki code .sbssI, html code.shiki .sbssI{--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C}html pre.shiki code .sfNiH, html code.shiki .sfNiH{--shiki-light:#FF5370;--shiki-default:#FF9CAC;--shiki-dark:#FF9CAC}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sfazB, html code.shiki .sfazB{--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D}",{"title":51,"searchDepth":75,"depth":75,"links":1510},[1511,1512,1513,1514,1515,1516,1517,1518],{"id":35,"depth":75,"text":36},{"id":430,"depth":75,"text":431},{"id":901,"depth":75,"text":902},{"id":969,"depth":75,"text":970},{"id":1020,"depth":75,"text":1021},{"id":1342,"depth":75,"text":1343},{"id":1437,"depth":75,"text":1438},{"id":1464,"depth":75,"text":1465},"2026-07-10","A token bucket in memory is trivial. Put it behind Redis and it works, until it doesn't. This walks through rate limiting from one node, to a shared Redis, to why that model collapses at millions of requests per second, and the shift Uber made: enforce locally, coordinate globally, and drop by probability.",false,"md",null,{},"\u002Fbackend\u002Fapi-design\u002Frate-limiting-at-scale",{"title":5,"description":1520},"rate-limiting-at-scale-redis-vs-probabilistic","2.backend\u002F2.api-design\u002F3.rate-limiting-at-scale",[1530,1531,1532,1533],"Rate-Limiting","Distributed-Systems","Go","Performance",[1535,1536,1537],"backend","platform-engineering","resilience","35U9_08BECy4KxLE-mHdBTq5XM3_XAc8yIlXvgkgUkU",{"id":1540,"title":1541,"body":1542,"date":1519,"description":2811,"draft":1521,"extension":1522,"img":1523,"meta":2812,"navigation":145,"path":2813,"seo":2814,"slug":1527,"stem":2815,"tags":2816,"topics":2817,"__hash__":2818},"content\u002F2.backend\u002F2.api-design\u002F3.rate-limiting-at-scale.fr.md","Pourquoi le rate limiting sur Redis casse à grande échelle (et ce que fait Uber à la place)",{"type":7,"value":1543,"toc":2801},[1544,1547,1554,1562,1566,1572,1840,1846,1850,1853,1856,2046,2058,2242,2245,2249,2255,2258,2288,2303,2310,2314,2325,2328,2342,2345,2351,2357,2361,2368,2373,2380,2624,2634,2641,2645,2727,2730,2734,2737,2747,2757,2761,2796,2799],[10,1545,1546],{},"Le rate limiting sur une seule machine est un problème résolu. Vous gardez un compteur, vous le rechargez sur un timer, vous rejetez les requêtes quand il tombe à zéro. Vingt lignes de code et c'est terminé.",[10,1548,1549,1550,1553],{},"Les ennuis commencent dès que vous avez plus d'une machine. « 100 requêtes par seconde » devient une affirmation sur ",[17,1551,1552],{},"toute la flotte",", pas sur un seul processus. Chaque nœud ne voit que la part du trafic qui l'a atteint, mais la limite est globale. D'un coup il vous faut de la coordination, et la coordination à grande échelle, c'est là que la plupart des conceptions s'effondrent en silence.",[10,1555,1556,1557,1561],{},"Cet article suit le même chemin qu'un vrai système : on démarre avec le token bucket en mémoire, on le met derrière Redis pour qu'il fonctionne entre plusieurs nœuds, on regarde ce modèle casser sous la charge, puis on observe ce qu'",[25,1558,1560],{"href":27,"rel":1559},[29],"Uber a construit pour son Global Rate Limiter"," à la place. L'objectif, c'est le raisonnement, pas une bibliothèque à copier-coller, pour que vous puissiez faire le bon choix pour vos propres services.",[33,1563,1565],{"id":1564},"étape-un-le-token-bucket-sur-un-seul-nœud","Étape un : le token bucket sur un seul nœud",[10,1567,1568,1569,1571],{},"Le token bucket est le classique. Un seau contient jusqu'à ",[41,1570,43],{}," jetons. Chaque requête prend un jeton. Les jetons se rechargent à un rythme constant. Si le seau est vide, la requête est rejetée. La capacité est ce qui vous permet d'absorber les pics courts ; le rythme de recharge, c'est votre limite en régime permanent.",[46,1573,1575],{"className":48,"code":1574,"language":50,"meta":51,"style":51},"type TokenBucket struct {\n    mu         sync.Mutex\n    tokens     float64\n    capacity   float64\n    refillRate float64 \u002F\u002F jetons par seconde\n    lastRefill time.Time\n}\n\nfunc (b *TokenBucket) Allow() bool {\n    b.mu.Lock()\n    defer b.mu.Unlock()\n\n    now := time.Now()\n    elapsed := now.Sub(b.lastRefill).Seconds()\n    b.tokens = math.Min(b.capacity, b.tokens+elapsed*b.refillRate)\n    b.lastRefill = now\n\n    if b.tokens >= 1 {\n        b.tokens--\n        return true\n    }\n    return false\n}\n",[41,1576,1577,1587,1597,1603,1609,1618,1628,1632,1636,1658,1672,1688,1692,1706,1732,1778,1790,1794,1810,1820,1826,1830,1836],{"__ignoreMap":51},[55,1578,1579,1581,1583,1585],{"class":57,"line":58},[55,1580,62],{"class":61},[55,1582,66],{"class":65},[55,1584,69],{"class":61},[55,1586,72],{"class":61},[55,1588,1589,1591,1593,1595],{"class":57,"line":75},[55,1590,79],{"class":78},[55,1592,82],{"class":65},[55,1594,85],{"class":61},[55,1596,88],{"class":65},[55,1598,1599,1601],{"class":57,"line":91},[55,1600,94],{"class":78},[55,1602,98],{"class":97},[55,1604,1605,1607],{"class":57,"line":101},[55,1606,104],{"class":78},[55,1608,98],{"class":97},[55,1610,1611,1613,1615],{"class":57,"line":109},[55,1612,112],{"class":78},[55,1614,115],{"class":97},[55,1616,1617],{"class":118}," \u002F\u002F jetons par seconde\n",[55,1619,1620,1622,1624,1626],{"class":57,"line":122},[55,1621,125],{"class":78},[55,1623,128],{"class":65},[55,1625,85],{"class":61},[55,1627,133],{"class":65},[55,1629,1630],{"class":57,"line":136},[55,1631,139],{"class":61},[55,1633,1634],{"class":57,"line":142},[55,1635,146],{"emptyLinePlaceholder":145},[55,1637,1638,1640,1642,1644,1646,1648,1650,1652,1654,1656],{"class":57,"line":149},[55,1639,152],{"class":61},[55,1641,155],{"class":61},[55,1643,159],{"class":158},[55,1645,162],{"class":61},[55,1647,165],{"class":65},[55,1649,168],{"class":61},[55,1651,172],{"class":171},[55,1653,175],{"class":61},[55,1655,178],{"class":97},[55,1657,72],{"class":61},[55,1659,1660,1662,1664,1666,1668,1670],{"class":57,"line":183},[55,1661,186],{"class":78},[55,1663,85],{"class":61},[55,1665,191],{"class":78},[55,1667,85],{"class":61},[55,1669,196],{"class":171},[55,1671,199],{"class":61},[55,1673,1674,1676,1678,1680,1682,1684,1686],{"class":57,"line":202},[55,1675,206],{"class":205},[55,1677,209],{"class":78},[55,1679,85],{"class":61},[55,1681,191],{"class":78},[55,1683,85],{"class":61},[55,1685,218],{"class":171},[55,1687,199],{"class":61},[55,1689,1690],{"class":57,"line":223},[55,1691,146],{"emptyLinePlaceholder":145},[55,1693,1694,1696,1698,1700,1702,1704],{"class":57,"line":228},[55,1695,231],{"class":78},[55,1697,234],{"class":61},[55,1699,237],{"class":78},[55,1701,85],{"class":61},[55,1703,242],{"class":171},[55,1705,199],{"class":61},[55,1707,1708,1710,1712,1714,1716,1718,1720,1722,1724,1726,1728,1730],{"class":57,"line":247},[55,1709,250],{"class":78},[55,1711,234],{"class":61},[55,1713,255],{"class":78},[55,1715,85],{"class":61},[55,1717,260],{"class":171},[55,1719,263],{"class":61},[55,1721,266],{"class":78},[55,1723,85],{"class":61},[55,1725,271],{"class":78},[55,1727,274],{"class":61},[55,1729,277],{"class":171},[55,1731,199],{"class":61},[55,1733,1734,1736,1738,1740,1742,1744,1746,1748,1750,1752,1754,1756,1758,1760,1762,1764,1766,1768,1770,1772,1774,1776],{"class":57,"line":282},[55,1735,186],{"class":78},[55,1737,85],{"class":61},[55,1739,289],{"class":78},[55,1741,292],{"class":61},[55,1743,295],{"class":78},[55,1745,85],{"class":61},[55,1747,300],{"class":171},[55,1749,263],{"class":61},[55,1751,266],{"class":78},[55,1753,85],{"class":61},[55,1755,43],{"class":78},[55,1757,311],{"class":61},[55,1759,209],{"class":78},[55,1761,85],{"class":61},[55,1763,318],{"class":78},[55,1765,321],{"class":61},[55,1767,324],{"class":78},[55,1769,162],{"class":61},[55,1771,266],{"class":78},[55,1773,85],{"class":61},[55,1775,333],{"class":78},[55,1777,336],{"class":61},[55,1779,1780,1782,1784,1786,1788],{"class":57,"line":339},[55,1781,186],{"class":78},[55,1783,85],{"class":61},[55,1785,346],{"class":78},[55,1787,292],{"class":61},[55,1789,351],{"class":78},[55,1791,1792],{"class":57,"line":354},[55,1793,146],{"emptyLinePlaceholder":145},[55,1795,1796,1798,1800,1802,1804,1806,1808],{"class":57,"line":359},[55,1797,362],{"class":205},[55,1799,209],{"class":78},[55,1801,85],{"class":61},[55,1803,289],{"class":78},[55,1805,371],{"class":61},[55,1807,375],{"class":374},[55,1809,72],{"class":61},[55,1811,1812,1814,1816,1818],{"class":57,"line":380},[55,1813,383],{"class":78},[55,1815,85],{"class":61},[55,1817,318],{"class":78},[55,1819,390],{"class":61},[55,1821,1822,1824],{"class":57,"line":393},[55,1823,396],{"class":205},[55,1825,400],{"class":399},[55,1827,1828],{"class":57,"line":403},[55,1829,406],{"class":61},[55,1831,1832,1834],{"class":57,"line":409},[55,1833,412],{"class":205},[55,1835,415],{"class":399},[55,1837,1838],{"class":57,"line":418},[55,1839,139],{"class":61},[10,1841,1842,1843,1845],{},"C'est correct, rapide, et le verrou est local. Pas de réseau, pas de dépendance, des décisions en nanosecondes. Il a exactement un défaut : le compteur vit dans la mémoire de ce processus. Lancez dix réplicas derrière un load balancer et vous avez dix seaux indépendants, chacun appliquant la limite entière. Votre vraie limite globale est maintenant ",[41,1844,426],{},". Personne n'a demandé ça.",[33,1847,1849],{"id":1848},"étape-deux-déplacer-létat-dans-redis","Étape deux : déplacer l'état dans Redis",[10,1851,1852],{},"La correction évidente, c'est de sortir le compteur du processus et de le mettre dans un store que chaque nœud peut voir. Redis est le choix par défaut : rapide, atomique, et tout le monde en fait déjà tourner.",[10,1854,1855],{},"La version naïve, c'est un compteur par fenêtre avec un TTL :",[46,1857,1859],{"className":48,"code":1858,"language":50,"meta":51,"style":51},"func Allow(ctx context.Context, rdb *redis.Client, key string, limit int, window time.Duration) (bool, error) {\n    \u002F\u002F clé du type \"rl:user:42:1720512000\", bucketée par fenêtre fixe\n    count, err := rdb.Incr(ctx, key).Result()\n    if err != nil {\n        return false, err\n    }\n    if count == 1 {\n        rdb.Expire(ctx, key, window)\n    }\n    return count \u003C= int64(limit), nil\n}\n",[41,1860,1861,1925,1930,1960,1972,1982,1986,1998,2020,2024,2042],{"__ignoreMap":51},[55,1862,1863,1865,1867,1869,1871,1873,1875,1877,1879,1881,1883,1885,1887,1889,1891,1893,1895,1897,1899,1901,1903,1905,1907,1909,1911,1913,1915,1917,1919,1921,1923],{"class":57,"line":58},[55,1864,152],{"class":61},[55,1866,172],{"class":171},[55,1868,263],{"class":61},[55,1870,453],{"class":158},[55,1872,456],{"class":65},[55,1874,85],{"class":61},[55,1876,461],{"class":65},[55,1878,311],{"class":61},[55,1880,466],{"class":158},[55,1882,469],{"class":61},[55,1884,472],{"class":65},[55,1886,85],{"class":61},[55,1888,477],{"class":65},[55,1890,311],{"class":61},[55,1892,482],{"class":158},[55,1894,485],{"class":97},[55,1896,311],{"class":61},[55,1898,490],{"class":158},[55,1900,493],{"class":97},[55,1902,311],{"class":61},[55,1904,498],{"class":158},[55,1906,237],{"class":65},[55,1908,85],{"class":61},[55,1910,505],{"class":65},[55,1912,168],{"class":61},[55,1914,155],{"class":61},[55,1916,512],{"class":97},[55,1918,311],{"class":61},[55,1920,517],{"class":97},[55,1922,168],{"class":61},[55,1924,72],{"class":61},[55,1926,1927],{"class":57,"line":75},[55,1928,1929],{"class":118},"    \u002F\u002F clé du type \"rl:user:42:1720512000\", bucketée par fenêtre fixe\n",[55,1931,1932,1934,1936,1938,1940,1942,1944,1946,1948,1950,1952,1954,1956,1958],{"class":57,"line":91},[55,1933,531],{"class":78},[55,1935,311],{"class":61},[55,1937,536],{"class":78},[55,1939,234],{"class":61},[55,1941,466],{"class":78},[55,1943,85],{"class":61},[55,1945,545],{"class":171},[55,1947,263],{"class":61},[55,1949,453],{"class":78},[55,1951,311],{"class":61},[55,1953,482],{"class":78},[55,1955,274],{"class":61},[55,1957,558],{"class":171},[55,1959,199],{"class":61},[55,1961,1962,1964,1966,1968,1970],{"class":57,"line":101},[55,1963,362],{"class":205},[55,1965,536],{"class":78},[55,1967,569],{"class":61},[55,1969,572],{"class":61},[55,1971,72],{"class":61},[55,1973,1974,1976,1978,1980],{"class":57,"line":109},[55,1975,396],{"class":205},[55,1977,581],{"class":399},[55,1979,311],{"class":61},[55,1981,586],{"class":78},[55,1983,1984],{"class":57,"line":122},[55,1985,406],{"class":61},[55,1987,1988,1990,1992,1994,1996],{"class":57,"line":136},[55,1989,362],{"class":205},[55,1991,597],{"class":78},[55,1993,600],{"class":61},[55,1995,375],{"class":374},[55,1997,72],{"class":61},[55,1999,2000,2002,2004,2006,2008,2010,2012,2014,2016,2018],{"class":57,"line":142},[55,2001,609],{"class":78},[55,2003,85],{"class":61},[55,2005,614],{"class":171},[55,2007,263],{"class":61},[55,2009,453],{"class":78},[55,2011,311],{"class":61},[55,2013,482],{"class":78},[55,2015,311],{"class":61},[55,2017,498],{"class":78},[55,2019,336],{"class":61},[55,2021,2022],{"class":57,"line":149},[55,2023,406],{"class":61},[55,2025,2026,2028,2030,2032,2034,2036,2038,2040],{"class":57,"line":183},[55,2027,412],{"class":205},[55,2029,597],{"class":78},[55,2031,641],{"class":61},[55,2033,644],{"class":97},[55,2035,263],{"class":61},[55,2037,649],{"class":78},[55,2039,652],{"class":61},[55,2041,655],{"class":61},[55,2043,2044],{"class":57,"line":202},[55,2045,139],{"class":61},[10,2047,2048,2049,2051,2052,2054,2055,2057],{},"Les fenêtres fixes ont un bord bien connu : un appelant peut envoyer ",[41,2050,649],{}," requêtes à la fin d'une fenêtre et ",[41,2053,649],{}," de plus au début de la suivante, soit ",[41,2056,671],{}," en un peu plus d'une seconde. La réponse habituelle est une fenêtre glissante dans un script Lua, exécuté atomiquement à l'intérieur de Redis :",[46,2059,2061],{"className":675,"code":2060,"language":677,"meta":51,"style":51},"-- KEYS[1] = clé du bucket, ARGV = now (ms), window (ms), limit\nlocal now    = tonumber(ARGV[1])\nlocal window = tonumber(ARGV[2])\nlocal limit  = tonumber(ARGV[3])\n\nredis.call('ZREMRANGEBYSCORE', KEYS[1], 0, now - window)\nlocal count = redis.call('ZCARD', KEYS[1])\nif count \u003C limit then\n  redis.call('ZADD', KEYS[1], now, now)\n  redis.call('PEXPIRE', KEYS[1], window)\n  return 1\nend\nreturn 0\n",[41,2062,2063,2068,2084,2100,2116,2120,2148,2174,2186,2206,2226,2232,2236],{"__ignoreMap":51},[55,2064,2065],{"class":57,"line":58},[55,2066,2067],{"class":118},"-- KEYS[1] = clé du bucket, ARGV = now (ms), window (ms), limit\n",[55,2069,2070,2072,2074,2076,2078,2080,2082],{"class":57,"line":75},[55,2071,689],{"class":61},[55,2073,692],{"class":78},[55,2075,292],{"class":61},[55,2077,697],{"class":171},[55,2079,700],{"class":78},[55,2081,703],{"class":374},[55,2083,706],{"class":78},[55,2085,2086,2088,2090,2092,2094,2096,2098],{"class":57,"line":91},[55,2087,689],{"class":61},[55,2089,713],{"class":78},[55,2091,292],{"class":61},[55,2093,697],{"class":171},[55,2095,700],{"class":78},[55,2097,722],{"class":374},[55,2099,706],{"class":78},[55,2101,2102,2104,2106,2108,2110,2112,2114],{"class":57,"line":101},[55,2103,689],{"class":61},[55,2105,731],{"class":78},[55,2107,292],{"class":61},[55,2109,697],{"class":171},[55,2111,700],{"class":78},[55,2113,740],{"class":374},[55,2115,706],{"class":78},[55,2117,2118],{"class":57,"line":109},[55,2119,146],{"emptyLinePlaceholder":145},[55,2121,2122,2124,2126,2128,2130,2132,2134,2136,2138,2140,2142,2144,2146],{"class":57,"line":122},[55,2123,751],{"class":78},[55,2125,754],{"class":171},[55,2127,263],{"class":78},[55,2129,759],{"class":61},[55,2131,763],{"class":762},[55,2133,759],{"class":61},[55,2135,768],{"class":78},[55,2137,703],{"class":374},[55,2139,773],{"class":78},[55,2141,776],{"class":374},[55,2143,779],{"class":78},[55,2145,782],{"class":61},[55,2147,785],{"class":78},[55,2149,2150,2152,2154,2156,2158,2160,2162,2164,2166,2168,2170,2172],{"class":57,"line":136},[55,2151,689],{"class":61},[55,2153,597],{"class":78},[55,2155,292],{"class":61},[55,2157,796],{"class":78},[55,2159,754],{"class":171},[55,2161,263],{"class":78},[55,2163,759],{"class":61},[55,2165,805],{"class":762},[55,2167,759],{"class":61},[55,2169,768],{"class":78},[55,2171,703],{"class":374},[55,2173,706],{"class":78},[55,2175,2176,2178,2180,2182,2184],{"class":57,"line":142},[55,2177,818],{"class":205},[55,2179,597],{"class":78},[55,2181,823],{"class":61},[55,2183,826],{"class":78},[55,2185,829],{"class":205},[55,2187,2188,2190,2192,2194,2196,2198,2200,2202,2204],{"class":57,"line":149},[55,2189,834],{"class":78},[55,2191,754],{"class":171},[55,2193,263],{"class":78},[55,2195,759],{"class":61},[55,2197,843],{"class":762},[55,2199,759],{"class":61},[55,2201,768],{"class":78},[55,2203,703],{"class":374},[55,2205,852],{"class":78},[55,2207,2208,2210,2212,2214,2216,2218,2220,2222,2224],{"class":57,"line":183},[55,2209,834],{"class":78},[55,2211,754],{"class":171},[55,2213,263],{"class":78},[55,2215,759],{"class":61},[55,2217,865],{"class":762},[55,2219,759],{"class":61},[55,2221,768],{"class":78},[55,2223,703],{"class":374},[55,2225,874],{"class":78},[55,2227,2228,2230],{"class":57,"line":202},[55,2229,879],{"class":205},[55,2231,882],{"class":374},[55,2233,2234],{"class":57,"line":223},[55,2235,887],{"class":205},[55,2237,2238,2240],{"class":57,"line":228},[55,2239,892],{"class":205},[55,2241,895],{"class":374},[10,2243,2244],{},"C'est précis. Chaque nœud voit le même état, la limite est vraiment globale, et la fenêtre glissante ferme le trou aux frontières. Pour beaucoup de systèmes, c'est la bonne réponse et vous pouvez vous arrêter là.",[33,2246,2248],{"id":2247},"étape-trois-là-où-le-modèle-redis-casse","Étape trois : là où le modèle Redis casse",[10,2250,2251,2252],{},"Le design Redis a une propriété qui semble inoffensive et devient fatale : ",[907,2253,2254],{},"chaque requête, sans exception, fait un aller-retour réseau avant que vous sachiez s'il faut l'autoriser.",[10,2256,2257],{},"Suivez ce que ça coûte à mesure que vous grandissez.",[914,2259,2260,2270,2276,2282],{},[917,2261,2262,2265,2266,2269],{},[907,2263,2264],{},"De la latence dans le chemin critique."," Chaque requête attend maintenant Redis avant de continuer. Un aller-retour, c'est une milliseconde ou deux un bon jour. Vous venez de l'ajouter au p50 de ",[17,2267,2268],{},"chaque"," endpoint, et la queue est pire quand Redis est chargé.",[917,2271,2272,2275],{},[907,2273,2274],{},"Une dépendance partagée dans le chemin critique."," Si Redis est lent, chaque service rate-limité est lent. Si Redis est en panne, soit vous échouez en mode ouvert (plus de limitation), soit en mode fermé (tout rejeter). Vous avez couplé la disponibilité de chaque service à un seul store.",[917,2277,2278,2281],{},[907,2279,2280],{},"Les clés chaudes (hot keys)."," Un appelant populaire ou un endpoint chaud dirige tout son trafic vers une seule clé, donc un seul shard Redis. Vous ne pouvez pas vous en sortir en shardant ; le sharding découpe les clés, pas la charge sur une clé unique.",[917,2283,2284,2287],{},[907,2285,2286],{},"La taxe de l'état global."," Maintenir un compteur global exact et en temps réel signifie que chaque nœud parle en permanence au même état de référence. L'équipe d'Uber a constaté qu'à leur volume ce n'était tout simplement pas viable, estimant que « des centaines de clusters Redis seraient nécessaires pour maintenir un état global exact en temps réel ».",[10,2289,2290,2291,2294,2295,2298,2299,2302],{},"Les chiffres qui font casser ça ne sont pas subtils. Uber tourne à l'ordre de ",[907,2292,2293],{},"80 millions de requêtes par seconde",", sur ",[907,2296,2297],{},"plus de 1 100 services"," et des centaines de milliers de hôtes. À cette échelle, un aller-retour par requête vers un store partagé n'est pas une taxe que vous pouvez vous offrir. Le store ",[17,2300,2301],{},"est"," le goulot d'étranglement.",[10,2304,2305,2306,2309],{},"Le problème racine, c'est l'exigence elle-même : ",[907,2307,2308],{},"un état global parfaitement exact, de façon synchrone, à chaque requête."," Si vous y tenez, vous êtes coincé à le payer. Alors remettez l'exigence en question.",[33,2311,2313],{"id":2312},"le-virage-appliquer-localement-coordonner-globalement","Le virage : appliquer localement, coordonner globalement",[10,2315,2316,2317,2320,2321,2324],{},"Voici l'idée qui débloque l'échelle. Vous n'avez pas besoin du compte global exact à chaque requête. Vous avez besoin que chaque nœud prenne une ",[17,2318,2319],{},"bonne"," décision locale, en s'appuyant sur une image globale ",[17,2322,2323],{},"légèrement périmée",". Un rate limit est une soupape de sécurité, pas un grand livre comptable. Si l'application retarde le trafic réel d'une seconde ou deux, presque rien ne casse.",[10,2326,2327],{},"Séparez donc les deux tâches que Redis faisait en même temps :",[986,2329,2330,2336],{},[917,2331,2332,2335],{},[907,2333,2334],{},"L'application"," se fait localement, dans le chemin critique, sans aucun appel réseau. Rapide.",[917,2337,2338,2341],{},[907,2339,2340],{},"La coordination"," se fait hors bande, de façon asynchrone. Les nœuds rapportent ce qu'ils voient ; un control plane agrège et leur dit à quel point ils doivent freiner.",[10,2343,2344],{},"Le Global Rate Limiter d'Uber est une boucle de rétroaction à trois niveaux bâtie exactement sur cette séparation :",[46,2346,2349],{"className":2347,"code":2348,"language":1008},[1006],"                                        ┌───────────────────────┐\n                                        │  Contrôleur global \u002F  │  agrège l'usage des zones,\n                                        │  régional             │  calcule les drop ratios,\n                                        └───────────▲───────────┘  pousse les directives vers le bas\n                                            usage   │   drop ratio\n                                                    │\n                                        ┌───────────┴───────────┐\n                                        │  Agrégateurs de zone  │  somment les comptes par hôte\n                                        └───────────▲───────────┘  en usage de zone\n                                            comptes │   drop ratio\n                                                    │\n                            ┌───────────────┬───────┴───────┬───────────────┐\n                            │ Data plane    │ Data plane    │ Data plane    │  applique localement,\n                            │ (proxy mesh)  │ (proxy mesh)  │ (proxy mesh)  │  rapporte les comptes\n                            └───────────────┴───────────────┴───────────────┘\n                                ▲ décision dans le chemin critique, sans saut réseau\n",[41,2350,2348],{"__ignoreMap":51},[10,2352,2353,2354,2356],{},"Les proxies décident chaque requête localement. En arrière-plan ils rapportent les comptes par hôte vers les agrégateurs de zone, qui remontent vers les contrôleurs, qui calculent à quel point chaque bucket est en surcharge et repoussent un seul nombre vers le bas : le ",[907,2355,1016],{},". Toute la boucle se referme en quelques secondes.",[33,2358,2360],{"id":2359},"lalgorithme-le-rejet-probabiliste","L'algorithme : le rejet probabiliste",[10,2362,2363,2364,2367],{},"Une fois qu'un nœud a un drop ratio du control plane, l'application devient presque triviale, et c'est la partie à bien intégrer. Au lieu de compter vers une limite dure, chaque nœud rejette un ",[17,2365,2366],{},"pourcentage"," de requêtes. Le pourcentage est calculé à partir de l'écart entre la flotte et sa limite :",[46,2369,2371],{"className":2370,"code":1032,"language":1008},[1006],[41,2372,1032],{"__ignoreMap":51},[10,2374,2375,2376,2379],{},"Si la flotte tourne à 1,5× sa limite, c'est ",[41,2377,2378],{},"(150 - 100) \u002F 150 ≈ 0,33",", donc chaque nœud rejette environ un tiers de ses requêtes. Additionnez les survivants sur tous les nœuds et vous retombez près de la limite, sans qu'aucun nœud ne compte jamais l'état global.",[46,2381,2383],{"className":48,"code":2382,"language":50,"meta":51,"style":51},"type Limiter struct {\n    dropRatio atomic.Value \u002F\u002F float64, mis à jour par lz boucle du control plane\n}\n\n\u002F\u002F Allow est appelé dans le chemin critique. Pas de réseau, pas de verrou sur le chemin rapide.\nfunc (l *Limiter) Allow() bool {\n    ratio, _ := l.dropRatio.Load().(float64)\n    if ratio \u003C= 0 {\n        return true \u002F\u002F la flotte est sous sa limite, on laisse tout passer\n    }\n    \u002F\u002F rejette chaque requête indépendamment avec la probabilité = ratio\n    return rand.Float64() >= ratio\n}\n\n\u002F\u002F mis à jour de façon asynchrone, hors du chemin des requêtes, chaque ~seconde\nfunc (l *Limiter) SetDropRatio(actualRPS, limitRPS float64) {\n    ratio := 0.0\n    if actualRPS > limitRPS {\n        ratio = (actualRPS - limitRPS) \u002F actualRPS\n    }\n    l.dropRatio.Store(ratio)\n}\n",[41,2384,2385,2395,2408,2412,2416,2421,2443,2469,2481,2490,2494,2499,2515,2519,2523,2528,2558,2566,2578,2598,2602,2620],{"__ignoreMap":51},[55,2386,2387,2389,2391,2393],{"class":57,"line":58},[55,2388,62],{"class":61},[55,2390,1053],{"class":65},[55,2392,69],{"class":61},[55,2394,72],{"class":61},[55,2396,2397,2399,2401,2403,2405],{"class":57,"line":75},[55,2398,1062],{"class":78},[55,2400,1065],{"class":65},[55,2402,85],{"class":61},[55,2404,1070],{"class":65},[55,2406,2407],{"class":118}," \u002F\u002F float64, mis à jour par lz boucle du control plane\n",[55,2409,2410],{"class":57,"line":91},[55,2411,139],{"class":61},[55,2413,2414],{"class":57,"line":101},[55,2415,146],{"emptyLinePlaceholder":145},[55,2417,2418],{"class":57,"line":109},[55,2419,2420],{"class":118},"\u002F\u002F Allow est appelé dans le chemin critique. Pas de réseau, pas de verrou sur le chemin rapide.\n",[55,2422,2423,2425,2427,2429,2431,2433,2435,2437,2439,2441],{"class":57,"line":122},[55,2424,152],{"class":61},[55,2426,155],{"class":61},[55,2428,1095],{"class":158},[55,2430,162],{"class":61},[55,2432,1100],{"class":65},[55,2434,168],{"class":61},[55,2436,172],{"class":171},[55,2438,175],{"class":61},[55,2440,178],{"class":97},[55,2442,72],{"class":61},[55,2444,2445,2447,2449,2451,2453,2455,2457,2459,2461,2463,2465,2467],{"class":57,"line":136},[55,2446,1115],{"class":78},[55,2448,311],{"class":61},[55,2450,1120],{"class":78},[55,2452,234],{"class":61},[55,2454,1125],{"class":78},[55,2456,85],{"class":61},[55,2458,1130],{"class":78},[55,2460,85],{"class":61},[55,2462,1135],{"class":171},[55,2464,1138],{"class":61},[55,2466,115],{"class":97},[55,2468,336],{"class":61},[55,2470,2471,2473,2475,2477,2479],{"class":57,"line":142},[55,2472,362],{"class":205},[55,2474,1149],{"class":78},[55,2476,641],{"class":61},[55,2478,1154],{"class":374},[55,2480,72],{"class":61},[55,2482,2483,2485,2487],{"class":57,"line":149},[55,2484,396],{"class":205},[55,2486,1163],{"class":399},[55,2488,2489],{"class":118}," \u002F\u002F la flotte est sous sa limite, on laisse tout passer\n",[55,2491,2492],{"class":57,"line":183},[55,2493,406],{"class":61},[55,2495,2496],{"class":57,"line":202},[55,2497,2498],{"class":118},"    \u002F\u002F rejette chaque requête indépendamment avec la probabilité = ratio\n",[55,2500,2501,2503,2505,2507,2509,2511,2513],{"class":57,"line":223},[55,2502,412],{"class":205},[55,2504,1182],{"class":78},[55,2506,85],{"class":61},[55,2508,1187],{"class":171},[55,2510,175],{"class":61},[55,2512,1192],{"class":61},[55,2514,1195],{"class":78},[55,2516,2517],{"class":57,"line":228},[55,2518,139],{"class":61},[55,2520,2521],{"class":57,"line":247},[55,2522,146],{"emptyLinePlaceholder":145},[55,2524,2525],{"class":57,"line":282},[55,2526,2527],{"class":118},"\u002F\u002F mis à jour de façon asynchrone, hors du chemin des requêtes, chaque ~seconde\n",[55,2529,2530,2532,2534,2536,2538,2540,2542,2544,2546,2548,2550,2552,2554,2556],{"class":57,"line":339},[55,2531,152],{"class":61},[55,2533,155],{"class":61},[55,2535,1095],{"class":158},[55,2537,162],{"class":61},[55,2539,1100],{"class":65},[55,2541,168],{"class":61},[55,2543,1225],{"class":171},[55,2545,263],{"class":61},[55,2547,1230],{"class":158},[55,2549,311],{"class":61},[55,2551,1235],{"class":158},[55,2553,1238],{"class":97},[55,2555,168],{"class":61},[55,2557,72],{"class":61},[55,2559,2560,2562,2564],{"class":57,"line":354},[55,2561,1247],{"class":78},[55,2563,234],{"class":61},[55,2565,1252],{"class":374},[55,2567,2568,2570,2572,2574,2576],{"class":57,"line":359},[55,2569,362],{"class":205},[55,2571,1259],{"class":78},[55,2573,1262],{"class":61},[55,2575,1265],{"class":78},[55,2577,1268],{"class":61},[55,2579,2580,2582,2584,2586,2588,2590,2592,2594,2596],{"class":57,"line":380},[55,2581,1273],{"class":78},[55,2583,292],{"class":61},[55,2585,155],{"class":61},[55,2587,1280],{"class":78},[55,2589,782],{"class":61},[55,2591,1235],{"class":78},[55,2593,168],{"class":61},[55,2595,1289],{"class":61},[55,2597,1292],{"class":78},[55,2599,2600],{"class":57,"line":393},[55,2601,406],{"class":61},[55,2603,2604,2606,2608,2610,2612,2614,2616,2618],{"class":57,"line":403},[55,2605,1301],{"class":78},[55,2607,85],{"class":61},[55,2609,1130],{"class":78},[55,2611,85],{"class":61},[55,2613,1310],{"class":171},[55,2615,263],{"class":61},[55,2617,1315],{"class":78},[55,2619,336],{"class":61},[55,2621,2622],{"class":57,"line":409},[55,2623,139],{"class":61},[10,2625,2626,2627,2629,2630,2633],{},"Regardez ce que ça nous rapporte. ",[41,2628,1327],{}," est un seul load atomique et un nombre aléatoire : pas de contention de verrou, pas de Redis, pas de clé chaude. La partie coûteuse, le calcul du ratio, a quitté le chemin des requêtes entièrement et tourne une fois par seconde sur des données agrégées. Le passage d'Uber à ce modèle a réduit drastiquement les latences de queue, avec un p99.5 qui chute jusqu'à ",[907,2631,2632],{},"90 %"," une fois l'aller-retour Redis disparu.",[10,2635,2636,2637,2640],{},"Le piège est honnête et mérite d'être dit : comme le drop ratio repose sur des données agrégées chaque seconde, l'application peut retarder le trafic réel de ",[907,2638,2639],{},"2 à 3 secondes",". Pour un pic soudain et extrêmement bref, le système réagit avec un temps de retard. En pratique cette fenêtre importe rarement, et l'échanger est précisément ce qui fait tenir l'échelle.",[33,2642,2644],{"id":2643},"les-compromis-côte-à-côte","Les compromis, côte à côte",[1345,2646,2647,2659],{},[1348,2648,2649],{},[1351,2650,2651,2653,2656],{},[1354,2652],{},[1354,2654,2655],{},"Redis centralisé",[1354,2657,2658],{},"Application locale + coordination globale",[1363,2660,2661,2672,2683,2694,2705,2716],{},[1351,2662,2663,2666,2669],{},[1368,2664,2665],{},"Chemin de décision",[1368,2667,2668],{},"Aller-retour réseau par requête",[1368,2670,2671],{},"Local, en mémoire",[1351,2673,2674,2677,2680],{},[1368,2675,2676],{},"Précision",[1368,2678,2679],{},"Exacte, temps réel",[1368,2681,2682],{},"Approximative, ~1 à 3s de retard",[1351,2684,2685,2688,2691],{},[1368,2686,2687],{},"Mode de panne",[1368,2689,2690],{},"Redis est un SPOF partagé",[1368,2692,2693],{},"Échoue en mode ouvert par nœud, pas de dépendance partagée",[1351,2695,2696,2699,2702],{},[1368,2697,2698],{},"Clés chaudes",[1368,2700,2701],{},"Convergent vers un shard",[1368,2703,2704],{},"Aucune, pas de compteur partagé",[1351,2706,2707,2710,2713],{},[1368,2708,2709],{},"Latence ajoutée",[1368,2711,2712],{},"1 à 2ms+ par requête",[1368,2714,2715],{},"Quasi nulle",[1351,2717,2718,2721,2724],{},[1368,2719,2720],{},"Passe à l'échelle jusqu'à",[1368,2722,2723],{},"Milliers de RPS confortablement",[1368,2725,2726],{},"Dizaines de millions de RPS",[10,2728,2729],{},"Il n'y a pas de repas gratuit ici, seulement un échange : vous abandonnez la précision exacte en temps réel et vous gagnez la capacité d'appliquer des limites à une échelle où le modèle exact ne peut physiquement pas tourner.",[33,2731,2733],{"id":2732},"deux-détails-qui-rendent-le-système-prêt-pour-la-production","Deux détails qui rendent le système prêt pour la production",[10,2735,2736],{},"L'algorithme central est le titre, mais deux pièces opérationnelles sont ce qui permet à une équipe de vraiment lui faire confiance.",[10,2738,2739,2742,2743,2746],{},[907,2740,2741],{},"Échouer en mode ouvert."," Si le control plane devient silencieux, les nœuds continuent de servir le trafic avec le dernier ratio qu'ils tenaient, ou aucun. Un rate limiter qui met tout le système à terre quand ",[17,2744,2745],{},"lui"," tombe est pire que pas de rate limiter du tout. L'application locale est ce qui rend ça sûr : un nœud n'a besoin de rien d'externe pour continuer à répondre.",[10,2748,2749,2752,2753,2756],{},[907,2750,2751],{},"Le mode shadow."," Avant qu'une nouvelle limite n'applique quoi que ce soit, faites-la tourner en shadow : calculez ce qui ",[17,2754,2755],{},"aurait"," été rejeté et émettez-le comme métrique, ne rejetez rien. Les équipes regardent le graphe, confirment que la limite est saine face au trafic réel, et seulement ensuite basculent en mode application. Uber couple ça à un réglage automatisé qui dérive les limites de plusieurs semaines de pics observés plus une marge, pour que les limites suivent le trafic au lieu de pourrir dans un YAML que quelqu'un a réglé il y a deux ans.",[33,2758,2760],{"id":2759},"ce-quil-faut-retenir","Ce qu'il faut retenir",[914,2762,2763,2769,2775,2781,2790],{},[917,2764,2765,2768],{},[907,2766,2767],{},"Le rate limiting sur un seul nœud est facile ; tout le problème est la coordination."," Chaque décision de conception est en réalité une décision sur la quantité de précision dans cette coordination que vous acceptez d'échanger contre l'échelle.",[917,2770,2771,2774],{},[907,2772,2773],{},"Un store partagé dans le chemin critique est un plafond de scalabilité."," Ça marche à merveille jusqu'à ce que l'aller-retour par requête, et le store lui-même, deviennent le goulot. Sachez où se trouve ce plafond pour votre trafic.",[917,2776,2777,2780],{},[907,2778,2779],{},"Relâcher « exact et temps réel » vers « approximatif et légèrement périmé » change tout le design."," Un retard de 1 à 3 secondes vous permet de déplacer l'application en local et de supprimer la dépendance partagée entièrement.",[917,2782,2783,2786,2787,2789],{},[907,2784,2785],{},"Le rejet probabiliste transforme un compte global difficile en une décision locale triviale :"," rejeter avec la probabilité ",[41,2788,1494],{},", calculée hors bande.",[917,2791,2792,2795],{},[907,2793,2794],{},"La limitation au niveau infrastructure bat celle au niveau applicatif en haut de la courbe d'échelle,"," parce qu'elle applique uniformément à travers chaque service sans que chacun ne réimplémente le même compteur.",[10,2797,2798],{},"Si vous tournez à des milliers de requêtes par seconde, la fenêtre glissante Redis est probablement le bon outil et vous ne devriez pas sur-concevoir. Mais si vous fixez le point où un aller-retour par requête cesse d'être abordable, la réponse n'est pas un Redis plus gros. C'est d'arrêter de demander un compte globalement exact à chaque requête, et de laisser chaque nœud décider par lui-même avec un nombre qui est assez bon.",[1506,2800,1508],{},{"title":51,"searchDepth":75,"depth":75,"links":2802},[2803,2804,2805,2806,2807,2808,2809,2810],{"id":1564,"depth":75,"text":1565},{"id":1848,"depth":75,"text":1849},{"id":2247,"depth":75,"text":2248},{"id":2312,"depth":75,"text":2313},{"id":2359,"depth":75,"text":2360},{"id":2643,"depth":75,"text":2644},{"id":2732,"depth":75,"text":2733},{"id":2759,"depth":75,"text":2760},"Un token bucket en mémoire, c'est trivial. Mettez-le derrière Redis et ça marche, jusqu'à ce que ça ne marche plus. On part du rate limiting sur un seul nœud, puis sur un Redis partagé, on voit pourquoi ce modèle s'effondre à des millions de requêtes par seconde, et le virage qu'a pris Uber : appliquer localement, coordonner globalement, et rejeter par probabilité.",{},"\u002Fbackend\u002Fapi-design\u002Frate-limiting-at-scale.fr",{"title":1541,"description":2811},"2.backend\u002F2.api-design\u002F3.rate-limiting-at-scale.fr",[1530,1531,1532,1533],[1535,1536,1537],"2v81fGq-3QqF6dA-frNm-S5ZfSboqfXwlqI9TfyYJAw",[2820,2823],{"title":2821,"path":2822},"Protobuf FieldMask for Writes: One Update Endpoint, No Ambiguity","\u002Fbackend\u002Fapi-design\u002Fprotobuf-fieldmask-mutations",{"title":2824,"path":2825},"Retry Storms: How Good Clients Take Down Healthy Servers","\u002Fbackend\u002Fapi-design\u002Fretry-storms",[2827,2834,2844],{"path":2825,"title":2824,"description":2828,"date":2829,"tags":2830,"topics":2833},"A retry looks harmless: the request failed, so try again. Multiply that by every client, add one slow dependency, and retries turn into a self-inflicted DDoS. This walks from the naive retry loop to exponential backoff, jitter, retry budgets and circuit breakers, the caller-side half of resilience that pairs with rate limiting on the server.","2026-07-11",[2831,1531,1532,2832],"Resilience","Retries",[1535,1536,1537],{"path":2835,"title":2836,"description":2837,"date":2838,"tags":2839,"topics":2843},"\u002Fbackend\u002Fapi-design\u002Fprotobuf-fieldmask","Protobuf FieldMask: Let Your API Clients Ask for Only What They Need","One endpoint, many callers, some very expensive fields. Protobuf FieldMask lets each client declare exactly which fields it wants, so your server can skip the costly database queries, remote calls and payload it doesn't need. The idea, a classic example, and the traps.","2026-07-03",[2840,2841,2842,1533],"gRPC","Protobuf","API-Design",[1535],{"path":2822,"title":2821,"description":2845,"date":2846,"tags":2847,"topics":2849},"A single Update method that changes only the fields you name, with no endpoint per field and no clobbering the rest. How the update_mask brings clean PATCH semantics to gRPC: setting, clearing, nested paths, and the empty-mask trap that can silently wipe data.","2026-07-07",[2840,2841,2842,2848],"Mutations",[1535],1783842138168]