Why Redis Rate Limiting Breaks at Scale (and What Uber Does Instead)

Rate limiting on a single machine is a solved problem. You keep a counter, you refill it on a timer, you reject requests when it hits zero. Twenty lines of code and you are done.

The trouble starts the moment you have more than one machine. Now "100 requests per second" is a statement about the whole fleet, not about any one process. Each node only sees the slice of traffic that reached it, but the limit is global. Suddenly you need coordination, and coordination at scale is where most designs quietly fall apart.

This article walks the same path a real system walks: start with the in-memory token bucket, put it behind Redis so it works across nodes, watch that model break under load, and then look at what Uber built for their Global Rate Limiter instead. The goal is the reasoning, not a copy-paste library, so you can make the right call for your own services.

Step one: the token bucket on one node

The token bucket is the classic. A bucket holds up to capacity tokens. Every request takes one token. Tokens refill at a steady rate. If the bucket is empty, the request is rejected. The capacity is what lets you absorb short bursts; the refill rate is your steady-state limit.

type TokenBucket struct {
    mu         sync.Mutex
    tokens     float64
    capacity   float64
    refillRate float64 // tokens per second
    lastRefill time.Time
}

func (b *TokenBucket) Allow() bool {
    b.mu.Lock()
    defer b.mu.Unlock()

    now := time.Now()
    elapsed := now.Sub(b.lastRefill).Seconds()
    b.tokens = math.Min(b.capacity, b.tokens+elapsed*b.refillRate)
    b.lastRefill = now

    if b.tokens >= 1 {
        b.tokens--
        return true
    }
    return false
}

This is correct, fast, and lock-local. No network, no dependency, decisions in nanoseconds. It has exactly one flaw: the counter lives in this process's memory. Run ten replicas behind a load balancer and you have ten independent buckets, each enforcing the full limit. Your real global limit is now 10 × capacity. Nobody asked for that.

Step two: move the state to Redis

The obvious fix is to pull the counter out of the process and into a store every node can see. Redis is the default choice: it is fast, atomic, and everyone already runs it.

The naive version is a counter per window with a TTL:

func Allow(ctx context.Context, rdb *redis.Client, key string, limit int, window time.Duration) (bool, error) {
    // key like "rl:user:42:1720512000", bucketed by fixed window
    count, err := rdb.Incr(ctx, key).Result()
    if err != nil {
        return false, err
    }
    if count == 1 {
        rdb.Expire(ctx, key, window)
    }
    return count <= int64(limit), nil
}

Fixed windows have a well-known edge: a caller can send limit requests at the end of one window and limit more at the start of the next, so 2 × limit in a hair over a second. The usual answer is a sliding window in a Lua script, run atomically inside Redis:

-- KEYS[1] = bucket key, ARGV = now (ms), window (ms), limit
local now    = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit  = tonumber(ARGV[3])

redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, now - window)
local count = redis.call('ZCARD', KEYS[1])
if count < limit then
  redis.call('ZADD', KEYS[1], now, now)
  redis.call('PEXPIRE', KEYS[1], window)
  return 1
end
return 0

This is accurate. Every node sees the same state, the limit is truly global, and the sliding window closes the boundary gap. For a lot of systems, this is the right answer and you can stop here.

Step three: where the Redis model breaks

The Redis design has one property that looks harmless and becomes fatal: every single request does a network round trip before you know whether to allow it.

Walk through what that costs as you grow.

• Latency in the hot path. Every request now waits on Redis before it proceeds. One round trip is a millisecond or two on a good day. You just added that to the p50 of every endpoint, and the tail is worse when Redis is busy.
• A shared dependency in the critical path. If Redis is slow, every rate-limited service is slow. If Redis is down, you either fail open (no limiting) or fail closed (reject everything). You have coupled the availability of every service to one store.
• Hot keys. A popular caller or a single hot endpoint funnels all its traffic to one key, which means one Redis shard. You cannot shard your way out of a single hot key; sharding splits keys, not the load on one key.
• The global-state tax. Keeping an accurate, real-time global counter means every node talks to the same authoritative state constantly. Uber's team found that at their volume this was simply not viable, estimating "hundreds of Redis clusters would be required to maintain accurate global state in real time."

The numbers that break this are not subtle. Uber runs on the order of 80 million requests per second, across 1,100+ services and hundreds of thousands of hosts. At that scale, one round trip per request to a shared store is not a tax you can afford. The store is the bottleneck.

The root problem is the requirement itself: perfectly accurate global state, synchronously, on every request. If you insist on that, you are stuck paying for it. So question the requirement.

The shift: enforce locally, coordinate globally

Here is the insight that unlocks scale. You do not need the exact global count on every request. You need each node to make a good local decision, using a global picture that is slightly stale. A rate limit is a safety valve, not a bank ledger. If enforcement lags real traffic by a second or two, almost nothing breaks.

So split the two jobs that Redis was doing at once:

1. Enforcement happens locally, in the hot path, with zero network calls. Fast.
2. Coordination happens out of band, asynchronously. Nodes report what they are seeing; a control plane aggregates it and tells them how hard to push back.

Uber's Global Rate Limiter is a three-tier feedback loop built on exactly this split:

                                                  ┌───────────────────────┐
                                                  │  Global / Regional    │  aggregates zone usage,
                                                  │  Controller           │  computes drop ratios,
                                                  └───────────▲───────────┘  pushes directives down
                                                      usage   │   drop ratio
                                                              │
                                                  ┌───────────┴───────────┐
                                                  │   Zone Aggregators    │  sum per-host counts
                                                  └───────────▲───────────┘  into zone usage
                                                      counts  │   drop ratio
                                                              │
                                      ┌───────────────┬───────┴───────┬───────────────┐
                                      │ Data plane    │ Data plane    │ Data plane    │  enforce locally,
                                      │ (mesh proxy)  │ (mesh proxy)  │ (mesh proxy)  │  report counts up
                                      └───────────────┴───────────────┴───────────────┘
                                          ▲ decision in the hot path, no network hop

The proxies decide every request locally. In the background they report per-host counts up to zone aggregators, which roll up to controllers, which compute how overloaded each bucket is and push a single number back down: the drop ratio. The whole loop closes in a couple of seconds.

The algorithm: probabilistic dropping

Once a node has a drop ratio from the control plane, enforcement becomes almost trivial, and this is the part worth internalizing. Instead of counting toward a hard limit, each node drops a percentage of requests. The percentage is computed from how far over the limit the fleet is running:

dropRatio = (actualRPS - limitRPS) / actualRPS

If the fleet is doing 1.5× its limit, that is (150 - 100) / 150 ≈ 0.33, so every node drops about a third of its requests. Add up the survivors across all nodes and you land back near the limit, without any node ever counting global state.

type Limiter struct {
    dropRatio atomic.Value // float64, updated by the control-plane loop
}

// Allow is called in the hot path. No network, no lock on the fast path.
func (l *Limiter) Allow() bool {
    ratio, _ := l.dropRatio.Load().(float64)
    if ratio <= 0 {
        return true // fleet is under its limit, let everything through
    }
    // drop each request independently with probability = ratio
    return rand.Float64() >= ratio
}

// updated asynchronously, out of the request path, every ~second
func (l *Limiter) SetDropRatio(actualRPS, limitRPS float64) {
    ratio := 0.0
    if actualRPS > limitRPS {
        ratio = (actualRPS - limitRPS) / actualRPS
    }
    l.dropRatio.Store(ratio)
}

Look at what this bought us. Allow() is a single atomic load and a random number: no lock contention, no Redis, no hot key. The expensive part, computing the ratio, moved out of the request path entirely and runs once a second on aggregated data. Uber's move to this model cut tail latencies dramatically, with p99.5 dropping by up to 90% once the Redis round trip was gone.

The catch is honest and worth stating: because the drop ratio rides on data aggregated every second, enforcement can lag real traffic by 2 to 3 seconds. For a sudden, extremely brief spike, the system reacts a beat late. In practice that window rarely matters, and trading it away is what makes the whole thing scale.

The tradeoffs, side by side

There is no free lunch here, only a swap: you give up exact real-time accuracy and gain the ability to enforce limits at a scale where the accurate model physically cannot run.

Two details that make it production-grade

The core algorithm is the headline, but two operational pieces are what let a team actually trust it.

Fail open. If the control plane goes quiet, nodes keep serving traffic with whatever ratio they last held, or none. A rate limiter that takes the whole system down when it fails is worse than no rate limiter. Enforcement being local is what makes this safe: a node needs nothing external to keep answering requests.

Shadow mode. Before a new limit enforces anything, run it in shadow: compute what would have been dropped and emit it as a metric, drop nothing. Teams watch the graph, confirm the limit is sane against real traffic, and only then flip to enforce. Uber pairs this with automated tuning that derives limits from weeks of observed peaks plus headroom, so limits track traffic instead of rotting in a YAML file someone set two years ago.

Takeaways

• Single-node rate limiting is easy; the whole problem is coordination. Every design decision is really a decision about how much accuracy in that coordination you are willing to trade for scale.
• A shared store on the hot path is a scaling ceiling. It works beautifully until the round trip per request, and the store itself, becomes the bottleneck. Know where that ceiling is for your traffic.
• Relaxing "exact and real-time" to "approximate and slightly stale" changes the whole design. A 1 to 3 second lag lets you move enforcement local and delete the shared dependency entirely.
• Probabilistic dropping turns a hard global count into a trivial local decision: drop with probability (actual − limit) / actual, computed out of band.
• Infrastructure-level limiting beats application-level at the top of the scale curve, because it enforces uniformly across every service without each one reimplementing the same counter.

If you are running thousands of requests per second, the Redis sliding window is probably the right tool and you should not overbuild. But if you are staring at the point where a round trip per request stops being affordable, the answer is not a bigger Redis. It is to stop asking for a globally accurate count on every request, and let each node decide for itself with a number that is good enough.