[{"data":1,"prerenderedAt":3765},["ShallowReactive",2],{"article-en-\u002Fbackend\u002Fload-balancing\u002Fload-balancing-algorithms":3,"article-sibling-en-\u002Fbackend\u002Fload-balancing\u002Fload-balancing-algorithms":2004,"surround-en-\u002Fbackend\u002Fload-balancing\u002Fload-balancing-algorithms":3730,"related-en-\u002Fbackend\u002Fload-balancing\u002Fload-balancing-algorithms":3737},{"id":4,"title":5,"body":6,"date":1984,"description":1985,"draft":1986,"extension":1987,"img":1988,"meta":1989,"navigation":98,"path":1990,"seo":1991,"slug":1992,"stem":1993,"tags":1994,"topics":1999,"__hash__":2003},"content\u002F2.backend\u002F3.load-balancing\u002F1.load-balancing-algorithms.md","Load Balancing Algorithms: Each One Fixes the Last One's Flaw",{"type":7,"value":8,"toc":1973},"minimark",[9,18,21,24,162,167,170,341,352,368,372,375,383,742,745,752,756,759,936,939,949,953,960,1142,1149,1152,1156,1159,1289,1304,1308,1311,1564,1579,1583,1590,1593,1613,1795,1820,1824,1924,1927,1931,1966,1969],[10,11,12,13,17],"p",{},"Spreading requests across a handful of servers sounds like the most trivial problem in distributed systems. You have three backends and a request; pick one, send it, done. A single call to ",[14,15,16],"code",{},"rand.Intn(3)"," and you have a load balancer. Ship it.",[10,19,20],{},"It works right up until reality shows up. One backend runs on a bigger machine than the others. One request takes five seconds while the rest take five milliseconds. One client needs to land on the same server every time because that is where its session lives. One backend quietly dies and your perfectly random balancer keeps cheerfully sending it a third of all traffic. Each of these is a different way the naive answer breaks, and each classic load balancing algorithm is a response to one specific break.",[10,22,23],{},"So this is not a list of six algorithms. It is a chain. We start with the simplest thing that works, watch it fail, and reach for the next algorithm precisely because it fixes what the last one could not. Go for the code, but the progression is the point. Everything hangs off one interface:",[25,26,31],"pre",{"className":27,"code":28,"language":29,"meta":30,"style":30},"language-go shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","\u002F\u002F A Balancer picks the backend that should serve the next request.\ntype Balancer interface {\n    Next(r *Request) *Backend\n}\n\ntype Backend struct {\n    URL         string\n    Weight      int          \u002F\u002F relative capacity, for weighted schemes\n    ActiveConns int64        \u002F\u002F in-flight requests, for least-connections\n    Healthy     bool\n}\n","go","",[14,32,33,42,59,87,93,100,113,124,136,148,157],{"__ignoreMap":30},[34,35,38],"span",{"class":36,"line":37},"line",1,[34,39,41],{"class":40},"sHwdD","\u002F\u002F A Balancer picks the backend that should serve the next request.\n",[34,43,45,49,53,56],{"class":36,"line":44},2,[34,46,48],{"class":47},"sMK4o","type",[34,50,52],{"class":51},"sBMFI"," Balancer",[34,54,55],{"class":47}," interface",[34,57,58],{"class":47}," {\n",[34,60,62,66,69,73,76,79,82,84],{"class":36,"line":61},3,[34,63,65],{"class":64},"s2Zo4","    Next",[34,67,68],{"class":47},"(",[34,70,72],{"class":71},"sHdIc","r",[34,74,75],{"class":47}," *",[34,77,78],{"class":51},"Request",[34,80,81],{"class":47},")",[34,83,75],{"class":47},[34,85,86],{"class":51},"Backend\n",[34,88,90],{"class":36,"line":89},4,[34,91,92],{"class":47},"}\n",[34,94,96],{"class":36,"line":95},5,[34,97,99],{"emptyLinePlaceholder":98},true,"\n",[34,101,103,105,108,111],{"class":36,"line":102},6,[34,104,48],{"class":47},[34,106,107],{"class":51}," Backend",[34,109,110],{"class":47}," struct",[34,112,58],{"class":47},[34,114,116,120],{"class":36,"line":115},7,[34,117,119],{"class":118},"sTEyZ","    URL         ",[34,121,123],{"class":122},"spNyl","string\n",[34,125,127,130,133],{"class":36,"line":126},8,[34,128,129],{"class":118},"    Weight      ",[34,131,132],{"class":122},"int",[34,134,135],{"class":40},"          \u002F\u002F relative capacity, for weighted schemes\n",[34,137,139,142,145],{"class":36,"line":138},9,[34,140,141],{"class":118},"    ActiveConns ",[34,143,144],{"class":122},"int64",[34,146,147],{"class":40},"        \u002F\u002F in-flight requests, for least-connections\n",[34,149,151,154],{"class":36,"line":150},10,[34,152,153],{"class":118},"    Healthy     ",[34,155,156],{"class":122},"bool\n",[34,158,160],{"class":36,"line":159},11,[34,161,92],{"class":47},[163,164,166],"h2",{"id":165},"round-robin-the-honest-starting-point","Round robin: the honest starting point",[10,168,169],{},"The first real algorithm past \"pick at random\" is round robin: hand each request to the next backend in order, wrapping around at the end. Every server gets an equal turn, in a predictable rotation.",[25,171,173],{"className":27,"code":172,"language":29,"meta":30,"style":30},"type RoundRobin struct {\n    backends []*Backend\n    counter  uint64\n}\n\nfunc (rr *RoundRobin) Next(_ *Request) *Backend {\n    \u002F\u002F atomic increment so concurrent callers each get a distinct slot\n    n := atomic.AddUint64(&rr.counter, 1)\n    return rr.backends[n%uint64(len(rr.backends))]\n}\n",[14,174,175,186,196,204,208,212,252,257,295,337],{"__ignoreMap":30},[34,176,177,179,182,184],{"class":36,"line":37},[34,178,48],{"class":47},[34,180,181],{"class":51}," RoundRobin",[34,183,110],{"class":47},[34,185,58],{"class":47},[34,187,188,191,194],{"class":36,"line":44},[34,189,190],{"class":118},"    backends ",[34,192,193],{"class":47},"[]*",[34,195,86],{"class":51},[34,197,198,201],{"class":36,"line":61},[34,199,200],{"class":118},"    counter  ",[34,202,203],{"class":122},"uint64\n",[34,205,206],{"class":36,"line":89},[34,207,92],{"class":47},[34,209,210],{"class":36,"line":95},[34,211,99],{"emptyLinePlaceholder":98},[34,213,214,217,220,223,226,229,231,234,236,239,241,243,245,247,250],{"class":36,"line":102},[34,215,216],{"class":47},"func",[34,218,219],{"class":47}," (",[34,221,222],{"class":71},"rr ",[34,224,225],{"class":47},"*",[34,227,228],{"class":51},"RoundRobin",[34,230,81],{"class":47},[34,232,233],{"class":64}," Next",[34,235,68],{"class":47},[34,237,238],{"class":71},"_",[34,240,75],{"class":47},[34,242,78],{"class":51},[34,244,81],{"class":47},[34,246,75],{"class":47},[34,248,249],{"class":51},"Backend",[34,251,58],{"class":47},[34,253,254],{"class":36,"line":115},[34,255,256],{"class":40},"    \u002F\u002F atomic increment so concurrent callers each get a distinct slot\n",[34,258,259,262,265,268,271,274,277,280,282,285,288,292],{"class":36,"line":126},[34,260,261],{"class":118},"    n ",[34,263,264],{"class":47},":=",[34,266,267],{"class":118}," atomic",[34,269,270],{"class":47},".",[34,272,273],{"class":64},"AddUint64",[34,275,276],{"class":47},"(&",[34,278,279],{"class":118},"rr",[34,281,270],{"class":47},[34,283,284],{"class":118},"counter",[34,286,287],{"class":47},",",[34,289,291],{"class":290},"sbssI"," 1",[34,293,294],{"class":47},")\n",[34,296,297,301,304,306,309,312,315,318,321,323,326,328,330,332,334],{"class":36,"line":138},[34,298,300],{"class":299},"s7zQu","    return",[34,302,303],{"class":118}," rr",[34,305,270],{"class":47},[34,307,308],{"class":118},"backends",[34,310,311],{"class":47},"[",[34,313,314],{"class":118},"n",[34,316,317],{"class":47},"%",[34,319,320],{"class":122},"uint64",[34,322,68],{"class":47},[34,324,325],{"class":64},"len",[34,327,68],{"class":47},[34,329,279],{"class":118},[34,331,270],{"class":47},[34,333,308],{"class":118},[34,335,336],{"class":47},"))]\n",[34,338,339],{"class":36,"line":150},[34,340,92],{"class":47},[10,342,343,344,347,348,351],{},"That ",[14,345,346],{},"atomic.AddUint64"," matters: a load balancer is hammered by many goroutines at once, and a plain ",[14,349,350],{},"counter++"," would race and hand two callers the same index. With the atomic, round robin is correct, lock-free, and about as cheap as dispatch gets.",[10,353,354,355,359,360,363,364,367],{},"Round robin makes one promise and keeps it: over time, every backend receives the same ",[356,357,358],"em",{},"number"," of requests. The trouble is that equal is not the same as fair. Round robin assumes every server is identical and every request costs the same, and both assumptions are usually false. Send a 16-core box and a 2-core box the same request count and you have overloaded one while the other idles. That gap between ",[356,361,362],{},"equal"," and ",[356,365,366],{},"fair"," is the flaw the next algorithm exists to fix.",[163,369,371],{"id":370},"weighted-round-robin-when-servers-are-not-equal","Weighted round robin: when servers are not equal",[10,373,374],{},"If one backend can handle three times the load of another, it should receive three times the traffic. Attach a weight to each server and distribute in proportion.",[10,376,377,378,382],{},"The obvious implementation is to expand the list, repeating each backend weight-many times, and round robin over that. It works, but it clusters: a server with weight 3 gets three requests in a burst, then nothing, which spikes load instead of smoothing it. The version real proxies use is ",[379,380,381],"strong",{},"smooth weighted round robin"," (this is Nginx's algorithm), which interleaves the picks so a weight-3 server is spread evenly through the rotation rather than bunched.",[25,384,386],{"className":27,"code":385,"language":29,"meta":30,"style":30},"type SmoothWRR struct {\n    backends []*Backend\n    current  []int \u002F\u002F running \"credit\" per backend\n    mu       sync.Mutex\n}\n\nfunc (w *SmoothWRR) Next(_ *Request) *Backend {\n    w.mu.Lock()\n    defer w.mu.Unlock()\n\n    total := 0\n    best := -1\n    for i, b := range w.backends {\n        w.current[i] += b.Weight \u002F\u002F each backend earns its weight in credit\n        total += b.Weight\n        if best == -1 || w.current[i] > w.current[best] {\n            best = i \u002F\u002F the most-owed backend wins this round\n        }\n    }\n    w.current[best] -= total \u002F\u002F and pays it back down\n    return w.backends[best]\n}\n",[14,387,388,399,407,420,433,437,441,475,493,512,516,526,540,569,602,618,668,683,689,695,719,737],{"__ignoreMap":30},[34,389,390,392,395,397],{"class":36,"line":37},[34,391,48],{"class":47},[34,393,394],{"class":51}," SmoothWRR",[34,396,110],{"class":47},[34,398,58],{"class":47},[34,400,401,403,405],{"class":36,"line":44},[34,402,190],{"class":118},[34,404,193],{"class":47},[34,406,86],{"class":51},[34,408,409,412,415,417],{"class":36,"line":61},[34,410,411],{"class":118},"    current  ",[34,413,414],{"class":47},"[]",[34,416,132],{"class":122},[34,418,419],{"class":40}," \u002F\u002F running \"credit\" per backend\n",[34,421,422,425,428,430],{"class":36,"line":89},[34,423,424],{"class":118},"    mu       ",[34,426,427],{"class":51},"sync",[34,429,270],{"class":47},[34,431,432],{"class":51},"Mutex\n",[34,434,435],{"class":36,"line":95},[34,436,92],{"class":47},[34,438,439],{"class":36,"line":102},[34,440,99],{"emptyLinePlaceholder":98},[34,442,443,445,447,450,452,455,457,459,461,463,465,467,469,471,473],{"class":36,"line":115},[34,444,216],{"class":47},[34,446,219],{"class":47},[34,448,449],{"class":71},"w ",[34,451,225],{"class":47},[34,453,454],{"class":51},"SmoothWRR",[34,456,81],{"class":47},[34,458,233],{"class":64},[34,460,68],{"class":47},[34,462,238],{"class":71},[34,464,75],{"class":47},[34,466,78],{"class":51},[34,468,81],{"class":47},[34,470,75],{"class":47},[34,472,249],{"class":51},[34,474,58],{"class":47},[34,476,477,480,482,485,487,490],{"class":36,"line":126},[34,478,479],{"class":118},"    w",[34,481,270],{"class":47},[34,483,484],{"class":118},"mu",[34,486,270],{"class":47},[34,488,489],{"class":64},"Lock",[34,491,492],{"class":47},"()\n",[34,494,495,498,501,503,505,507,510],{"class":36,"line":138},[34,496,497],{"class":299},"    defer",[34,499,500],{"class":118}," w",[34,502,270],{"class":47},[34,504,484],{"class":118},[34,506,270],{"class":47},[34,508,509],{"class":64},"Unlock",[34,511,492],{"class":47},[34,513,514],{"class":36,"line":150},[34,515,99],{"emptyLinePlaceholder":98},[34,517,518,521,523],{"class":36,"line":159},[34,519,520],{"class":118},"    total ",[34,522,264],{"class":47},[34,524,525],{"class":290}," 0\n",[34,527,529,532,534,537],{"class":36,"line":528},12,[34,530,531],{"class":118},"    best ",[34,533,264],{"class":47},[34,535,536],{"class":47}," -",[34,538,539],{"class":290},"1\n",[34,541,543,546,549,551,554,556,559,561,563,566],{"class":36,"line":542},13,[34,544,545],{"class":299},"    for",[34,547,548],{"class":118}," i",[34,550,287],{"class":47},[34,552,553],{"class":118}," b ",[34,555,264],{"class":47},[34,557,558],{"class":299}," range",[34,560,500],{"class":118},[34,562,270],{"class":47},[34,564,565],{"class":118},"backends ",[34,567,568],{"class":47},"{\n",[34,570,572,575,577,580,582,585,588,591,594,596,599],{"class":36,"line":571},14,[34,573,574],{"class":118},"        w",[34,576,270],{"class":47},[34,578,579],{"class":118},"current",[34,581,311],{"class":47},[34,583,584],{"class":118},"i",[34,586,587],{"class":47},"]",[34,589,590],{"class":47}," +=",[34,592,593],{"class":118}," b",[34,595,270],{"class":47},[34,597,598],{"class":118},"Weight ",[34,600,601],{"class":40},"\u002F\u002F each backend earns its weight in credit\n",[34,603,605,608,611,613,615],{"class":36,"line":604},15,[34,606,607],{"class":118},"        total ",[34,609,610],{"class":47},"+=",[34,612,593],{"class":118},[34,614,270],{"class":47},[34,616,617],{"class":118},"Weight\n",[34,619,621,624,627,630,632,635,638,640,642,644,646,648,650,653,655,657,659,661,664,666],{"class":36,"line":620},16,[34,622,623],{"class":299},"        if",[34,625,626],{"class":118}," best ",[34,628,629],{"class":47},"==",[34,631,536],{"class":47},[34,633,634],{"class":290},"1",[34,636,637],{"class":47}," ||",[34,639,500],{"class":118},[34,641,270],{"class":47},[34,643,579],{"class":118},[34,645,311],{"class":47},[34,647,584],{"class":118},[34,649,587],{"class":47},[34,651,652],{"class":47}," >",[34,654,500],{"class":118},[34,656,270],{"class":47},[34,658,579],{"class":118},[34,660,311],{"class":47},[34,662,663],{"class":118},"best",[34,665,587],{"class":47},[34,667,58],{"class":47},[34,669,671,674,677,680],{"class":36,"line":670},17,[34,672,673],{"class":118},"            best ",[34,675,676],{"class":47},"=",[34,678,679],{"class":118}," i ",[34,681,682],{"class":40},"\u002F\u002F the most-owed backend wins this round\n",[34,684,686],{"class":36,"line":685},18,[34,687,688],{"class":47},"        }\n",[34,690,692],{"class":36,"line":691},19,[34,693,694],{"class":47},"    }\n",[34,696,698,700,702,704,706,708,710,713,716],{"class":36,"line":697},20,[34,699,479],{"class":118},[34,701,270],{"class":47},[34,703,579],{"class":118},[34,705,311],{"class":47},[34,707,663],{"class":118},[34,709,587],{"class":47},[34,711,712],{"class":47}," -=",[34,714,715],{"class":118}," total ",[34,717,718],{"class":40},"\u002F\u002F and pays it back down\n",[34,720,722,724,726,728,730,732,734],{"class":36,"line":721},21,[34,723,300],{"class":299},[34,725,500],{"class":118},[34,727,270],{"class":47},[34,729,308],{"class":118},[34,731,311],{"class":47},[34,733,663],{"class":118},[34,735,736],{"class":47},"]\n",[34,738,740],{"class":36,"line":739},22,[34,741,92],{"class":47},[10,743,744],{},"Each backend accrues credit equal to its weight every round; the one with the most credit is chosen and then charged the total weight back. Over a full cycle a weight-3 server wins three times as often, but the wins are spread out, not clumped. The spikes are gone.",[10,746,747,748,751],{},"Weighted round robin fixes static capacity differences. What it cannot see is ",[356,749,750],{},"dynamic"," load. Weights are configured ahead of time and never change, so if a \"small\" server happens to catch a burst of slow requests, weighted round robin keeps feeding it at its configured rate, blind to the fact that it is drowning right now. The weights describe capacity in theory, not load in the moment.",[163,753,755],{"id":754},"least-connections-react-to-real-load","Least connections: react to real load",[10,757,758],{},"To respond to what is actually happening, stop counting turns and start counting work in flight. Least connections routes each request to the backend with the fewest active connections, so a server bogged down by slow requests naturally stops receiving new ones until it catches up.",[25,760,762],{"className":27,"code":761,"language":29,"meta":30,"style":30},"func (lc *LeastConn) Next(_ *Request) *Backend {\n    var best *Backend\n    min := int64(math.MaxInt64)\n    for _, b := range lc.backends {\n        c := atomic.LoadInt64(&b.ActiveConns)\n        if c \u003C min {\n            min, best = c, b\n        }\n    }\n    return best \u002F\u002F caller increments ActiveConns, decrements when the request finishes\n}\n",[14,763,764,798,809,831,855,881,896,915,919,923,932],{"__ignoreMap":30},[34,765,766,768,770,773,775,778,780,782,784,786,788,790,792,794,796],{"class":36,"line":37},[34,767,216],{"class":47},[34,769,219],{"class":47},[34,771,772],{"class":71},"lc ",[34,774,225],{"class":47},[34,776,777],{"class":51},"LeastConn",[34,779,81],{"class":47},[34,781,233],{"class":64},[34,783,68],{"class":47},[34,785,238],{"class":71},[34,787,75],{"class":47},[34,789,78],{"class":51},[34,791,81],{"class":47},[34,793,75],{"class":47},[34,795,249],{"class":51},[34,797,58],{"class":47},[34,799,800,803,805,807],{"class":36,"line":44},[34,801,802],{"class":47},"    var",[34,804,626],{"class":118},[34,806,225],{"class":47},[34,808,86],{"class":51},[34,810,811,814,816,819,821,824,826,829],{"class":36,"line":61},[34,812,813],{"class":118},"    min ",[34,815,264],{"class":47},[34,817,818],{"class":122}," int64",[34,820,68],{"class":47},[34,822,823],{"class":118},"math",[34,825,270],{"class":47},[34,827,828],{"class":118},"MaxInt64",[34,830,294],{"class":47},[34,832,833,835,838,840,842,844,846,849,851,853],{"class":36,"line":89},[34,834,545],{"class":299},[34,836,837],{"class":118}," _",[34,839,287],{"class":47},[34,841,553],{"class":118},[34,843,264],{"class":47},[34,845,558],{"class":299},[34,847,848],{"class":118}," lc",[34,850,270],{"class":47},[34,852,565],{"class":118},[34,854,568],{"class":47},[34,856,857,860,862,864,866,869,871,874,876,879],{"class":36,"line":95},[34,858,859],{"class":118},"        c ",[34,861,264],{"class":47},[34,863,267],{"class":118},[34,865,270],{"class":47},[34,867,868],{"class":64},"LoadInt64",[34,870,276],{"class":47},[34,872,873],{"class":118},"b",[34,875,270],{"class":47},[34,877,878],{"class":118},"ActiveConns",[34,880,294],{"class":47},[34,882,883,885,888,891,894],{"class":36,"line":102},[34,884,623],{"class":299},[34,886,887],{"class":118}," c ",[34,889,890],{"class":47},"\u003C",[34,892,893],{"class":118}," min ",[34,895,568],{"class":47},[34,897,898,901,903,905,907,910,912],{"class":36,"line":115},[34,899,900],{"class":118},"            min",[34,902,287],{"class":47},[34,904,626],{"class":118},[34,906,676],{"class":47},[34,908,909],{"class":118}," c",[34,911,287],{"class":47},[34,913,914],{"class":118}," b\n",[34,916,917],{"class":36,"line":126},[34,918,688],{"class":47},[34,920,921],{"class":36,"line":138},[34,922,694],{"class":47},[34,924,925,927,929],{"class":36,"line":150},[34,926,300],{"class":299},[34,928,626],{"class":118},[34,930,931],{"class":40},"\u002F\u002F caller increments ActiveConns, decrements when the request finishes\n",[34,933,934],{"class":36,"line":159},[34,935,92],{"class":47},[10,937,938],{},"This adapts to reality in a way round robin never can. Uneven request durations, a backend having a bad minute, a slow downstream: least connections routes around all of it automatically, because a struggling server accumulates connections and drops to the back of the queue. It is the first algorithm here that reacts to load instead of assuming it.",[10,940,941,942,945,946,948],{},"The cost is in that loop. To pick the minimum, least connections scans ",[356,943,944],{},"every"," backend on ",[356,947,944],{}," request, which is fine for a handful of servers and painful for hundreds. Worse, under high concurrency the read-then-route is racy: many goroutines all observe the same least-loaded backend at the same instant and stampede it together, the very imbalance the algorithm was meant to prevent. Reading global state to make a local decision does not scale.",[163,950,952],{"id":951},"power-of-two-choices-the-trick-nobody-teaches","Power of two choices: the trick nobody teaches",[10,954,955,956,959],{},"Here is the algorithm that should be far more famous than it is. Instead of scanning all backends for the true minimum, pick ",[379,957,958],{},"two backends at random"," and route to the less loaded of the two. That is the whole idea.",[25,961,963],{"className":27,"code":962,"language":29,"meta":30,"style":30},"func (p *P2C) Next(_ *Request) *Backend {\n    a := p.backends[rand.Intn(len(p.backends))]\n    b := p.backends[rand.Intn(len(p.backends))]\n    \u002F\u002F compare only these two, not the whole fleet\n    if atomic.LoadInt64(&a.ActiveConns) \u003C= atomic.LoadInt64(&b.ActiveConns) {\n        return a\n    }\n    return b\n}\n",[14,964,965,999,1037,1072,1077,1120,1128,1132,1138],{"__ignoreMap":30},[34,966,967,969,971,974,976,979,981,983,985,987,989,991,993,995,997],{"class":36,"line":37},[34,968,216],{"class":47},[34,970,219],{"class":47},[34,972,973],{"class":71},"p ",[34,975,225],{"class":47},[34,977,978],{"class":51},"P2C",[34,980,81],{"class":47},[34,982,233],{"class":64},[34,984,68],{"class":47},[34,986,238],{"class":71},[34,988,75],{"class":47},[34,990,78],{"class":51},[34,992,81],{"class":47},[34,994,75],{"class":47},[34,996,249],{"class":51},[34,998,58],{"class":47},[34,1000,1001,1004,1006,1009,1011,1013,1015,1018,1020,1023,1025,1027,1029,1031,1033,1035],{"class":36,"line":44},[34,1002,1003],{"class":118},"    a ",[34,1005,264],{"class":47},[34,1007,1008],{"class":118}," p",[34,1010,270],{"class":47},[34,1012,308],{"class":118},[34,1014,311],{"class":47},[34,1016,1017],{"class":118},"rand",[34,1019,270],{"class":47},[34,1021,1022],{"class":64},"Intn",[34,1024,68],{"class":47},[34,1026,325],{"class":64},[34,1028,68],{"class":47},[34,1030,10],{"class":118},[34,1032,270],{"class":47},[34,1034,308],{"class":118},[34,1036,336],{"class":47},[34,1038,1039,1042,1044,1046,1048,1050,1052,1054,1056,1058,1060,1062,1064,1066,1068,1070],{"class":36,"line":61},[34,1040,1041],{"class":118},"    b ",[34,1043,264],{"class":47},[34,1045,1008],{"class":118},[34,1047,270],{"class":47},[34,1049,308],{"class":118},[34,1051,311],{"class":47},[34,1053,1017],{"class":118},[34,1055,270],{"class":47},[34,1057,1022],{"class":64},[34,1059,68],{"class":47},[34,1061,325],{"class":64},[34,1063,68],{"class":47},[34,1065,10],{"class":118},[34,1067,270],{"class":47},[34,1069,308],{"class":118},[34,1071,336],{"class":47},[34,1073,1074],{"class":36,"line":89},[34,1075,1076],{"class":40},"    \u002F\u002F compare only these two, not the whole fleet\n",[34,1078,1079,1082,1084,1086,1088,1090,1093,1095,1097,1099,1102,1104,1106,1108,1110,1112,1114,1116,1118],{"class":36,"line":95},[34,1080,1081],{"class":299},"    if",[34,1083,267],{"class":118},[34,1085,270],{"class":47},[34,1087,868],{"class":64},[34,1089,276],{"class":47},[34,1091,1092],{"class":118},"a",[34,1094,270],{"class":47},[34,1096,878],{"class":118},[34,1098,81],{"class":47},[34,1100,1101],{"class":47}," \u003C=",[34,1103,267],{"class":118},[34,1105,270],{"class":47},[34,1107,868],{"class":64},[34,1109,276],{"class":47},[34,1111,873],{"class":118},[34,1113,270],{"class":47},[34,1115,878],{"class":118},[34,1117,81],{"class":47},[34,1119,58],{"class":47},[34,1121,1122,1125],{"class":36,"line":102},[34,1123,1124],{"class":299},"        return",[34,1126,1127],{"class":118}," a\n",[34,1129,1130],{"class":36,"line":115},[34,1131,694],{"class":47},[34,1133,1134,1136],{"class":36,"line":126},[34,1135,300],{"class":299},[34,1137,914],{"class":118},[34,1139,1140],{"class":36,"line":138},[34,1141,92],{"class":47},[10,1143,1144,1145,1148],{},"It looks too simple to be good, and the result is genuinely surprising: sampling just two and taking the better one gives you load distribution nearly as even as scanning the entire fleet, at O(1) cost instead of O(N). The math behind it (the \"power of two choices\" result) shows that the maximum load across servers drops exponentially compared to pure random, while a single extra sample buys almost all of that benefit. And because two goroutines rarely sample the ",[356,1146,1147],{},"same"," pair, the herding problem that plagued least connections mostly evaporates.",[10,1150,1151],{},"This is not a toy. Power of two choices is what production systems actually reach for: it is the default in HAProxy's newer versions, in Twitter's Finagle, in Envoy's load balancing. It is the sweet spot of the whole progression, near-global quality at local cost, and most tutorials skip straight past it. If you remember one algorithm from this article, remember this one.",[163,1153,1155],{"id":1154},"ip-hash-when-the-client-must-stick","IP hash: when the client must stick",[10,1157,1158],{},"Everything so far assumes any backend can serve any request. Sometimes that is false: a client has a session, a cache, or an upload in progress tied to one specific server, and it must return to the same one. The classic answer is to hash the client's IP and use it to pick a backend deterministically.",[25,1160,1162],{"className":27,"code":1161,"language":29,"meta":30,"style":30},"func (h *IPHash) Next(r *Request) *Backend {\n    sum := fnv.New32a()\n    sum.Write([]byte(r.ClientIP))\n    return h.backends[sum.Sum32()%uint32(len(h.backends))]\n}\n",[14,1163,1164,1198,1215,1243,1285],{"__ignoreMap":30},[34,1165,1166,1168,1170,1173,1175,1178,1180,1182,1184,1186,1188,1190,1192,1194,1196],{"class":36,"line":37},[34,1167,216],{"class":47},[34,1169,219],{"class":47},[34,1171,1172],{"class":71},"h ",[34,1174,225],{"class":47},[34,1176,1177],{"class":51},"IPHash",[34,1179,81],{"class":47},[34,1181,233],{"class":64},[34,1183,68],{"class":47},[34,1185,72],{"class":71},[34,1187,75],{"class":47},[34,1189,78],{"class":51},[34,1191,81],{"class":47},[34,1193,75],{"class":47},[34,1195,249],{"class":51},[34,1197,58],{"class":47},[34,1199,1200,1203,1205,1208,1210,1213],{"class":36,"line":44},[34,1201,1202],{"class":118},"    sum ",[34,1204,264],{"class":47},[34,1206,1207],{"class":118}," fnv",[34,1209,270],{"class":47},[34,1211,1212],{"class":64},"New32a",[34,1214,492],{"class":47},[34,1216,1217,1220,1222,1225,1228,1231,1233,1235,1237,1240],{"class":36,"line":61},[34,1218,1219],{"class":118},"    sum",[34,1221,270],{"class":47},[34,1223,1224],{"class":64},"Write",[34,1226,1227],{"class":47},"([]",[34,1229,1230],{"class":122},"byte",[34,1232,68],{"class":47},[34,1234,72],{"class":118},[34,1236,270],{"class":47},[34,1238,1239],{"class":118},"ClientIP",[34,1241,1242],{"class":47},"))\n",[34,1244,1245,1247,1250,1252,1254,1256,1259,1261,1264,1267,1270,1272,1274,1276,1279,1281,1283],{"class":36,"line":89},[34,1246,300],{"class":299},[34,1248,1249],{"class":118}," h",[34,1251,270],{"class":47},[34,1253,308],{"class":118},[34,1255,311],{"class":47},[34,1257,1258],{"class":118},"sum",[34,1260,270],{"class":47},[34,1262,1263],{"class":64},"Sum32",[34,1265,1266],{"class":47},"()%",[34,1268,1269],{"class":122},"uint32",[34,1271,68],{"class":47},[34,1273,325],{"class":64},[34,1275,68],{"class":47},[34,1277,1278],{"class":118},"h",[34,1280,270],{"class":47},[34,1282,308],{"class":118},[34,1284,336],{"class":47},[34,1286,1287],{"class":36,"line":95},[34,1288,92],{"class":47},[10,1290,1291,1292,1295,1296,1299,1300,1303],{},"Same IP, same hash, same backend, every time. Sessions stay pinned. It is clean and it works, and it hides a trap that only springs when you change the pool. That ",[14,1293,1294],{},"% len(backends)"," is the whole problem: the moment you add or remove a single server, ",[14,1297,1298],{},"N"," changes, and ",[356,1301,1302],{},"almost every"," client suddenly hashes to a different backend. Every session breaks, every cache goes cold, all at once, for a one-server change. Naive hashing is stable only as long as the pool never changes, which is not a property real systems have.",[163,1305,1307],{"id":1306},"consistent-hashing-stability-under-change","Consistent hashing: stability under change",[10,1309,1310],{},"The fix is to hash servers and clients onto the same abstract ring, a space of, say, 0 to 2^32, and walk clockwise from the client's position to the first server on the ring. Add or remove a server and only the keys in that server's arc move; everyone else stays put.",[25,1312,1314],{"className":27,"code":1313,"language":29,"meta":30,"style":30},"type Ring struct {\n    sorted []uint32          \u002F\u002F hash positions, sorted\n    owner  map[uint32]*Backend\n}\n\nfunc (r *Ring) Next(req *Request) *Backend {\n    h := hash(req.ClientIP)\n    \u002F\u002F binary search for the first ring position >= h, wrapping around\n    i := sort.Search(len(r.sorted), func(i int) bool { return r.sorted[i] >= h })\n    if i == len(r.sorted) {\n        i = 0 \u002F\u002F past the end of the ring, wrap to the start\n    }\n    return r.owner[r.sorted[i]]\n}\n",[14,1315,1316,1327,1339,1354,1358,1362,1397,1417,1422,1494,1517,1530,1534,1560],{"__ignoreMap":30},[34,1317,1318,1320,1323,1325],{"class":36,"line":37},[34,1319,48],{"class":47},[34,1321,1322],{"class":51}," Ring",[34,1324,110],{"class":47},[34,1326,58],{"class":47},[34,1328,1329,1332,1334,1336],{"class":36,"line":44},[34,1330,1331],{"class":118},"    sorted ",[34,1333,414],{"class":47},[34,1335,1269],{"class":122},[34,1337,1338],{"class":40},"          \u002F\u002F hash positions, sorted\n",[34,1340,1341,1344,1347,1349,1352],{"class":36,"line":61},[34,1342,1343],{"class":118},"    owner  ",[34,1345,1346],{"class":47},"map[",[34,1348,1269],{"class":122},[34,1350,1351],{"class":47},"]*",[34,1353,86],{"class":51},[34,1355,1356],{"class":36,"line":89},[34,1357,92],{"class":47},[34,1359,1360],{"class":36,"line":95},[34,1361,99],{"emptyLinePlaceholder":98},[34,1363,1364,1366,1368,1371,1373,1376,1378,1380,1382,1385,1387,1389,1391,1393,1395],{"class":36,"line":102},[34,1365,216],{"class":47},[34,1367,219],{"class":47},[34,1369,1370],{"class":71},"r ",[34,1372,225],{"class":47},[34,1374,1375],{"class":51},"Ring",[34,1377,81],{"class":47},[34,1379,233],{"class":64},[34,1381,68],{"class":47},[34,1383,1384],{"class":71},"req",[34,1386,75],{"class":47},[34,1388,78],{"class":51},[34,1390,81],{"class":47},[34,1392,75],{"class":47},[34,1394,249],{"class":51},[34,1396,58],{"class":47},[34,1398,1399,1402,1404,1407,1409,1411,1413,1415],{"class":36,"line":115},[34,1400,1401],{"class":118},"    h ",[34,1403,264],{"class":47},[34,1405,1406],{"class":64}," hash",[34,1408,68],{"class":47},[34,1410,1384],{"class":118},[34,1412,270],{"class":47},[34,1414,1239],{"class":118},[34,1416,294],{"class":47},[34,1418,1419],{"class":36,"line":126},[34,1420,1421],{"class":40},"    \u002F\u002F binary search for the first ring position >= h, wrapping around\n",[34,1423,1424,1427,1429,1432,1434,1437,1439,1441,1443,1445,1447,1450,1453,1456,1458,1461,1463,1466,1469,1472,1475,1477,1479,1481,1483,1485,1488,1491],{"class":36,"line":138},[34,1425,1426],{"class":118},"    i ",[34,1428,264],{"class":47},[34,1430,1431],{"class":118}," sort",[34,1433,270],{"class":47},[34,1435,1436],{"class":64},"Search",[34,1438,68],{"class":47},[34,1440,325],{"class":64},[34,1442,68],{"class":47},[34,1444,72],{"class":118},[34,1446,270],{"class":47},[34,1448,1449],{"class":118},"sorted",[34,1451,1452],{"class":47},"),",[34,1454,1455],{"class":47}," func(",[34,1457,584],{"class":71},[34,1459,1460],{"class":122}," int",[34,1462,81],{"class":47},[34,1464,1465],{"class":122}," bool",[34,1467,1468],{"class":47}," {",[34,1470,1471],{"class":299}," return",[34,1473,1474],{"class":118}," r",[34,1476,270],{"class":47},[34,1478,1449],{"class":118},[34,1480,311],{"class":47},[34,1482,584],{"class":118},[34,1484,587],{"class":47},[34,1486,1487],{"class":47}," >=",[34,1489,1490],{"class":118}," h ",[34,1492,1493],{"class":47},"})\n",[34,1495,1496,1498,1500,1502,1505,1507,1509,1511,1513,1515],{"class":36,"line":150},[34,1497,1081],{"class":299},[34,1499,679],{"class":118},[34,1501,629],{"class":47},[34,1503,1504],{"class":64}," len",[34,1506,68],{"class":47},[34,1508,72],{"class":118},[34,1510,270],{"class":47},[34,1512,1449],{"class":118},[34,1514,81],{"class":47},[34,1516,58],{"class":47},[34,1518,1519,1522,1524,1527],{"class":36,"line":159},[34,1520,1521],{"class":118},"        i ",[34,1523,676],{"class":47},[34,1525,1526],{"class":290}," 0",[34,1528,1529],{"class":40}," \u002F\u002F past the end of the ring, wrap to the start\n",[34,1531,1532],{"class":36,"line":528},[34,1533,694],{"class":47},[34,1535,1536,1538,1540,1542,1545,1547,1549,1551,1553,1555,1557],{"class":36,"line":542},[34,1537,300],{"class":299},[34,1539,1474],{"class":118},[34,1541,270],{"class":47},[34,1543,1544],{"class":118},"owner",[34,1546,311],{"class":47},[34,1548,72],{"class":118},[34,1550,270],{"class":47},[34,1552,1449],{"class":118},[34,1554,311],{"class":47},[34,1556,584],{"class":118},[34,1558,1559],{"class":47},"]]\n",[34,1561,1562],{"class":36,"line":571},[34,1563,92],{"class":47},[10,1565,1566,1567,1570,1571,1574,1575,1578],{},"With naive hashing, removing one of N servers remaps roughly ",[356,1568,1569],{},"all"," keys. With consistent hashing, it remaps only about ",[14,1572,1573],{},"K\u002FN"," of them, the ones that lived in the departed server's arc. In practice you also add ",[379,1576,1577],{},"virtual nodes",", hashing each backend to many ring positions instead of one, so load spreads evenly and no single server owns a giant arc by accident. This is the same structure behind distributed caches and hash-based sharding: consistent hashing is less a load balancing trick than a general answer to \"assign keys to nodes so that adding a node barely disturbs anything.\"",[163,1580,1582],{"id":1581},"the-failure-no-algorithm-fixes-dead-servers","The failure no algorithm fixes: dead servers",[10,1584,1585,1586,1589],{},"Step back and notice what every algorithm above quietly assumes: that the backend it picks is ",[356,1587,1588],{},"alive",". Round robin will dutifully route a third of your traffic to a server that died thirty seconds ago. Power of two choices will sample a corpse. Consistent hashing will pin a client to a backend that stopped answering. A flawless selection algorithm pointed at a dead server returns errors just as reliably as it would return successes.",[10,1591,1592],{},"So a real load balancer needs a layer none of these algorithms provide: knowing which backends are actually up. Two approaches, usually combined:",[1594,1595,1596,1607],"ul",{},[1597,1598,1599,1602,1603,1606],"li",{},[379,1600,1601],{},"Active health checks."," Periodically probe each backend (a ",[14,1604,1605],{},"GET \u002Fhealthz"," on a timer). Mark it unhealthy on failure, drop it from the pool, and keep probing so it can rejoin when it recovers.",[1597,1608,1609,1612],{},[379,1610,1611],{},"Passive health checks."," Watch real traffic. If a backend starts returning errors or timing out, eject it without waiting for the next probe. This is the load balancer's version of a circuit breaker.",[25,1614,1616],{"className":27,"code":1615,"language":29,"meta":30,"style":30},"func (rr *RoundRobin) Next(_ *Request) *Backend {\n    for i := 0; i \u003C len(rr.backends); i++ {\n        n := atomic.AddUint64(&rr.counter, 1)\n        b := rr.backends[n%uint64(len(rr.backends))]\n        if b.Healthy { \u002F\u002F skip ejected backends, try the next slot\n            return b\n        }\n    }\n    return nil \u002F\u002F whole pool is down; caller must shed or fail fast\n}\n",[14,1617,1618,1650,1687,1714,1749,1766,1773,1777,1781,1791],{"__ignoreMap":30},[34,1619,1620,1622,1624,1626,1628,1630,1632,1634,1636,1638,1640,1642,1644,1646,1648],{"class":36,"line":37},[34,1621,216],{"class":47},[34,1623,219],{"class":47},[34,1625,222],{"class":71},[34,1627,225],{"class":47},[34,1629,228],{"class":51},[34,1631,81],{"class":47},[34,1633,233],{"class":64},[34,1635,68],{"class":47},[34,1637,238],{"class":71},[34,1639,75],{"class":47},[34,1641,78],{"class":51},[34,1643,81],{"class":47},[34,1645,75],{"class":47},[34,1647,249],{"class":51},[34,1649,58],{"class":47},[34,1651,1652,1654,1656,1658,1660,1663,1665,1667,1669,1671,1673,1675,1677,1680,1682,1685],{"class":36,"line":44},[34,1653,545],{"class":299},[34,1655,679],{"class":118},[34,1657,264],{"class":47},[34,1659,1526],{"class":290},[34,1661,1662],{"class":47},";",[34,1664,679],{"class":118},[34,1666,890],{"class":47},[34,1668,1504],{"class":64},[34,1670,68],{"class":47},[34,1672,279],{"class":118},[34,1674,270],{"class":47},[34,1676,308],{"class":118},[34,1678,1679],{"class":47},");",[34,1681,548],{"class":118},[34,1683,1684],{"class":47},"++",[34,1686,58],{"class":47},[34,1688,1689,1692,1694,1696,1698,1700,1702,1704,1706,1708,1710,1712],{"class":36,"line":61},[34,1690,1691],{"class":118},"        n ",[34,1693,264],{"class":47},[34,1695,267],{"class":118},[34,1697,270],{"class":47},[34,1699,273],{"class":64},[34,1701,276],{"class":47},[34,1703,279],{"class":118},[34,1705,270],{"class":47},[34,1707,284],{"class":118},[34,1709,287],{"class":47},[34,1711,291],{"class":290},[34,1713,294],{"class":47},[34,1715,1716,1719,1721,1723,1725,1727,1729,1731,1733,1735,1737,1739,1741,1743,1745,1747],{"class":36,"line":89},[34,1717,1718],{"class":118},"        b ",[34,1720,264],{"class":47},[34,1722,303],{"class":118},[34,1724,270],{"class":47},[34,1726,308],{"class":118},[34,1728,311],{"class":47},[34,1730,314],{"class":118},[34,1732,317],{"class":47},[34,1734,320],{"class":122},[34,1736,68],{"class":47},[34,1738,325],{"class":64},[34,1740,68],{"class":47},[34,1742,279],{"class":118},[34,1744,270],{"class":47},[34,1746,308],{"class":118},[34,1748,336],{"class":47},[34,1750,1751,1753,1755,1757,1760,1763],{"class":36,"line":95},[34,1752,623],{"class":299},[34,1754,593],{"class":118},[34,1756,270],{"class":47},[34,1758,1759],{"class":118},"Healthy ",[34,1761,1762],{"class":47},"{",[34,1764,1765],{"class":40}," \u002F\u002F skip ejected backends, try the next slot\n",[34,1767,1768,1771],{"class":36,"line":102},[34,1769,1770],{"class":299},"            return",[34,1772,914],{"class":118},[34,1774,1775],{"class":36,"line":115},[34,1776,688],{"class":47},[34,1778,1779],{"class":36,"line":126},[34,1780,694],{"class":47},[34,1782,1783,1785,1788],{"class":36,"line":138},[34,1784,300],{"class":299},[34,1786,1787],{"class":47}," nil",[34,1789,1790],{"class":40}," \u002F\u002F whole pool is down; caller must shed or fail fast\n",[34,1792,1793],{"class":36,"line":150},[34,1794,92],{"class":47},[10,1796,1797,1798,1802,1803,1807,1808,1811,1812,1815,1816,1819],{},"This closes the loop with the rest of the resilience toolkit. Health-check ejection is the server-selection cousin of ",[1092,1799,1801],{"href":1800},"\u002Fblog\u002Fbackend\u002Fapi-design\u002Fretry-storms","the circuit breaker a client uses to stop calling a dead dependency",", and it sits alongside ",[1092,1804,1806],{"href":1805},"\u002Fblog\u002Fbackend\u002Fapi-design\u002Frate-limiting-at-scale","rate limiting that sheds load a service cannot serve",". Load balancing is the third face of the same coin: rate limiting decides ",[356,1809,1810],{},"how much"," a server takes, retries decide ",[356,1813,1814],{},"when"," a client sends, and load balancing decides ",[356,1817,1818],{},"which"," server gets it. Get the selection perfect and skip the health checks, and you have built a very fair way to distribute failure.",[163,1821,1823],{"id":1822},"the-chain-at-a-glance","The chain, at a glance",[1825,1826,1827,1843],"table",{},[1828,1829,1830],"thead",{},[1831,1832,1833,1837,1840],"tr",{},[1834,1835,1836],"th",{},"Algorithm",[1834,1838,1839],{},"The flaw it fixes",[1834,1841,1842],{},"What it still cannot do",[1844,1845,1846,1858,1869,1880,1891,1902,1913],"tbody",{},[1831,1847,1848,1852,1855],{},[1849,1850,1851],"td",{},"Round robin",[1849,1853,1854],{},"Random's clustering; gives strict equal turns",[1849,1856,1857],{},"Treats unequal servers and requests as equal",[1831,1859,1860,1863,1866],{},[1849,1861,1862],{},"Weighted round robin",[1849,1864,1865],{},"Servers with different capacity",[1849,1867,1868],{},"Weights are static, blind to live load",[1831,1870,1871,1874,1877],{},[1849,1872,1873],{},"Least connections",[1849,1875,1876],{},"Reacts to real, in-flight load",[1849,1878,1879],{},"O(N) scan per request; herds under concurrency",[1831,1881,1882,1885,1888],{},[1849,1883,1884],{},"Power of two choices",[1849,1886,1887],{},"Least-connections' cost and herding",[1849,1889,1890],{},"Assumes any backend can serve any request",[1831,1892,1893,1896,1899],{},[1849,1894,1895],{},"IP hash",[1849,1897,1898],{},"Session stickiness to one backend",[1849,1900,1901],{},"Rehashes everything when the pool changes",[1831,1903,1904,1907,1910],{},[1849,1905,1906],{},"Consistent hashing",[1849,1908,1909],{},"Stability when servers come and go",[1849,1911,1912],{},"Nothing, if you route to dead backends",[1831,1914,1915,1918,1921],{},[1849,1916,1917],{},"Health checks",[1849,1919,1920],{},"Routing to servers that are down",[1849,1922,1923],{},"n\u002Fa: this is the layer under all of them",[10,1925,1926],{},"None of these is \"the best.\" Each is the right answer to a specific pressure, and each was reached for because the previous one hit a wall.",[163,1928,1930],{"id":1929},"takeaways","Takeaways",[1594,1932,1933,1939,1945,1954,1960],{},[1597,1934,1935,1938],{},[379,1936,1937],{},"Start simpler than you think you need."," For a small fleet of similar servers with similar requests, round robin or even plain random is genuinely fine. Reach for more only when a real flaw bites.",[1597,1940,1941,1944],{},[379,1942,1943],{},"Equal is not fair."," Use weights when servers differ in capacity, and connection counts when requests differ in cost. Round robin conflates both and is blind to both.",[1597,1946,1947,1950,1951,1953],{},[379,1948,1949],{},"Power of two choices is the underrated sweet spot."," Near-global load quality at O(1) cost, with none of least-connections' scanning or herding. It is what production proxies actually use, and it is one ",[14,1952,1017],{}," call away.",[1597,1955,1956,1959],{},[379,1957,1958],{},"Naive hashing breaks the instant the pool changes."," If you need sticky routing, use consistent hashing with virtual nodes, or accept that adding one server will cold-cache all of them.",[1597,1961,1962,1965],{},[379,1963,1964],{},"The best selection algorithm still routes to dead servers."," Health checks and ejection are not optional polish; they are the layer that makes every algorithm above actually work in production.",[10,1967,1968],{},"The lesson of the whole chain is that \"load balancing\" is not one problem with one answer. It is a sequence of increasingly specific pressures, capacity, live load, cost, stickiness, churn, failure, and each classic algorithm is the tool that answers exactly one of them. Knowing which pressure you actually have is most of knowing which algorithm to pick.",[1970,1971,1972],"style",{},"html pre.shiki code .sHwdD, html code.shiki .sHwdD{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic}html pre.shiki code .sMK4o, html code.shiki .sMK4o{--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF}html pre.shiki code .sBMFI, html code.shiki .sBMFI{--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B}html pre.shiki code .s2Zo4, html code.shiki .s2Zo4{--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF}html pre.shiki code .sHdIc, html code.shiki .sHdIc{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#EEFFFF;--shiki-default-font-style:italic;--shiki-dark:#BABED8;--shiki-dark-font-style:italic}html pre.shiki code .sTEyZ, html code.shiki .sTEyZ{--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8}html pre.shiki code .spNyl, html code.shiki .spNyl{--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sbssI, html code.shiki .sbssI{--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C}html pre.shiki code .s7zQu, html code.shiki .s7zQu{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic}",{"title":30,"searchDepth":44,"depth":44,"links":1974},[1975,1976,1977,1978,1979,1980,1981,1982,1983],{"id":165,"depth":44,"text":166},{"id":370,"depth":44,"text":371},{"id":754,"depth":44,"text":755},{"id":951,"depth":44,"text":952},{"id":1154,"depth":44,"text":1155},{"id":1306,"depth":44,"text":1307},{"id":1581,"depth":44,"text":1582},{"id":1822,"depth":44,"text":1823},{"id":1929,"depth":44,"text":1930},"2026-07-20","Spreading requests across N servers sounds like a one-liner: pick a server, send the request. Then one backend is slower, or bigger, or holds a session, and the naive choice falls apart. This walks the classic algorithms as a chain where each one exists to fix the previous one's blind spot: round robin, weighted, least connections, power of two choices, and consistent hashing, in Go, ending with the failure every tutorial forgets.",false,"md",null,{},"\u002Fbackend\u002Fload-balancing\u002Fload-balancing-algorithms",{"title":5,"description":1985},"load-balancing-algorithms-go","2.backend\u002F3.load-balancing\u002F1.load-balancing-algorithms",[1995,1996,1997,1998],"Load-Balancing","Go","Distributed-Systems","Resilience",[2000,2001,2002],"backend","resilience","platform-engineering","hQGm4_Muw1ETRo8DUetMoXLwexblaxExb7f_iZy9vQQ",{"id":2005,"title":2006,"body":2007,"date":1984,"description":3721,"draft":1986,"extension":1987,"img":1988,"meta":3722,"navigation":98,"path":3723,"seo":3724,"slug":3725,"stem":3726,"tags":3727,"topics":3728,"__hash__":3729},"content\u002F2.backend\u002F3.load-balancing\u002F1.load-balancing-algorithms.fr.md","Algorithmes de load balancing : chacun repare le defaut du precedent",{"type":7,"value":2008,"toc":3710},[2009,2015,2018,2021,2111,2115,2118,2254,2263,2278,2282,2285,2291,2588,2591,2598,2602,2605,2763,2766,2776,2780,2787,2957,2964,2967,2971,2974,3086,3099,3103,3106,3330,3344,3348,3355,3358,3375,3548,3571,3575,3663,3666,3670,3705,3708],[10,2010,2011,2012,2014],{},"Repartir des requetes sur une poignee de serveurs sonne comme le probleme le plus trivial des systemes distribues. Vous avez trois backends et une requete ; en choisir un, l'envoyer, fini. Un seul appel a ",[14,2013,16],{}," et vous avez un load balancer. En production.",[10,2016,2017],{},"Ca marche jusqu'a ce que la realite se pointe. Un backend tourne sur une machine plus grosse que les autres. Une requete prend cinq secondes quand les autres prennent cinq millisecondes. Un client doit atterrir sur le meme serveur a chaque fois parce que c'est la que vit sa session. Un backend meurt discretement et votre balancer parfaitement aleatoire continue joyeusement de lui envoyer un tiers du trafic. Chacun de ces cas est une facon differente dont la reponse naive casse, et chaque algorithme classique de load balancing est une reponse a une casse precise.",[10,2019,2020],{},"Ceci n'est donc pas une liste de six algorithmes. C'est une chaine. On part de la chose la plus simple qui marche, on la regarde echouer, et on saisit l'algorithme suivant precisement parce qu'il repare ce que le precedent ne pouvait pas. Le Go pour le code, mais la progression est le sujet. Tout pend a une seule interface :",[25,2022,2024],{"className":27,"code":2023,"language":29,"meta":30,"style":30},"\u002F\u002F Un Balancer choisit le backend qui doit servir la prochaine requete.\ntype Balancer interface {\n    Next(r *Request) *Backend\n}\n\ntype Backend struct {\n    URL         string\n    Weight      int          \u002F\u002F capacite relative, pour les schemas ponderes\n    ActiveConns int64        \u002F\u002F requetes en cours, pour least-connections\n    Healthy     bool\n}\n",[14,2025,2026,2031,2041,2059,2063,2067,2077,2083,2092,2101,2107],{"__ignoreMap":30},[34,2027,2028],{"class":36,"line":37},[34,2029,2030],{"class":40},"\u002F\u002F Un Balancer choisit le backend qui doit servir la prochaine requete.\n",[34,2032,2033,2035,2037,2039],{"class":36,"line":44},[34,2034,48],{"class":47},[34,2036,52],{"class":51},[34,2038,55],{"class":47},[34,2040,58],{"class":47},[34,2042,2043,2045,2047,2049,2051,2053,2055,2057],{"class":36,"line":61},[34,2044,65],{"class":64},[34,2046,68],{"class":47},[34,2048,72],{"class":71},[34,2050,75],{"class":47},[34,2052,78],{"class":51},[34,2054,81],{"class":47},[34,2056,75],{"class":47},[34,2058,86],{"class":51},[34,2060,2061],{"class":36,"line":89},[34,2062,92],{"class":47},[34,2064,2065],{"class":36,"line":95},[34,2066,99],{"emptyLinePlaceholder":98},[34,2068,2069,2071,2073,2075],{"class":36,"line":102},[34,2070,48],{"class":47},[34,2072,107],{"class":51},[34,2074,110],{"class":47},[34,2076,58],{"class":47},[34,2078,2079,2081],{"class":36,"line":115},[34,2080,119],{"class":118},[34,2082,123],{"class":122},[34,2084,2085,2087,2089],{"class":36,"line":126},[34,2086,129],{"class":118},[34,2088,132],{"class":122},[34,2090,2091],{"class":40},"          \u002F\u002F capacite relative, pour les schemas ponderes\n",[34,2093,2094,2096,2098],{"class":36,"line":138},[34,2095,141],{"class":118},[34,2097,144],{"class":122},[34,2099,2100],{"class":40},"        \u002F\u002F requetes en cours, pour least-connections\n",[34,2102,2103,2105],{"class":36,"line":150},[34,2104,153],{"class":118},[34,2106,156],{"class":122},[34,2108,2109],{"class":36,"line":159},[34,2110,92],{"class":47},[163,2112,2114],{"id":2113},"round-robin-le-point-de-depart-honnete","Round robin : le point de depart honnete",[10,2116,2117],{},"Le premier vrai algorithme apres \"choisir au hasard\" est le round robin : donner chaque requete au backend suivant dans l'ordre, en repartant du debut a la fin. Chaque serveur a un tour egal, dans une rotation previsible.",[25,2119,2121],{"className":27,"code":2120,"language":29,"meta":30,"style":30},"type RoundRobin struct {\n    backends []*Backend\n    counter  uint64\n}\n\nfunc (rr *RoundRobin) Next(_ *Request) *Backend {\n    \u002F\u002F increment atomique pour que des appelants concurrents aient chacun un slot distinct\n    n := atomic.AddUint64(&rr.counter, 1)\n    return rr.backends[n%uint64(len(rr.backends))]\n}\n",[14,2122,2123,2133,2141,2147,2151,2155,2187,2192,2218,2250],{"__ignoreMap":30},[34,2124,2125,2127,2129,2131],{"class":36,"line":37},[34,2126,48],{"class":47},[34,2128,181],{"class":51},[34,2130,110],{"class":47},[34,2132,58],{"class":47},[34,2134,2135,2137,2139],{"class":36,"line":44},[34,2136,190],{"class":118},[34,2138,193],{"class":47},[34,2140,86],{"class":51},[34,2142,2143,2145],{"class":36,"line":61},[34,2144,200],{"class":118},[34,2146,203],{"class":122},[34,2148,2149],{"class":36,"line":89},[34,2150,92],{"class":47},[34,2152,2153],{"class":36,"line":95},[34,2154,99],{"emptyLinePlaceholder":98},[34,2156,2157,2159,2161,2163,2165,2167,2169,2171,2173,2175,2177,2179,2181,2183,2185],{"class":36,"line":102},[34,2158,216],{"class":47},[34,2160,219],{"class":47},[34,2162,222],{"class":71},[34,2164,225],{"class":47},[34,2166,228],{"class":51},[34,2168,81],{"class":47},[34,2170,233],{"class":64},[34,2172,68],{"class":47},[34,2174,238],{"class":71},[34,2176,75],{"class":47},[34,2178,78],{"class":51},[34,2180,81],{"class":47},[34,2182,75],{"class":47},[34,2184,249],{"class":51},[34,2186,58],{"class":47},[34,2188,2189],{"class":36,"line":115},[34,2190,2191],{"class":40},"    \u002F\u002F increment atomique pour que des appelants concurrents aient chacun un slot distinct\n",[34,2193,2194,2196,2198,2200,2202,2204,2206,2208,2210,2212,2214,2216],{"class":36,"line":126},[34,2195,261],{"class":118},[34,2197,264],{"class":47},[34,2199,267],{"class":118},[34,2201,270],{"class":47},[34,2203,273],{"class":64},[34,2205,276],{"class":47},[34,2207,279],{"class":118},[34,2209,270],{"class":47},[34,2211,284],{"class":118},[34,2213,287],{"class":47},[34,2215,291],{"class":290},[34,2217,294],{"class":47},[34,2219,2220,2222,2224,2226,2228,2230,2232,2234,2236,2238,2240,2242,2244,2246,2248],{"class":36,"line":138},[34,2221,300],{"class":299},[34,2223,303],{"class":118},[34,2225,270],{"class":47},[34,2227,308],{"class":118},[34,2229,311],{"class":47},[34,2231,314],{"class":118},[34,2233,317],{"class":47},[34,2235,320],{"class":122},[34,2237,68],{"class":47},[34,2239,325],{"class":64},[34,2241,68],{"class":47},[34,2243,279],{"class":118},[34,2245,270],{"class":47},[34,2247,308],{"class":118},[34,2249,336],{"class":47},[34,2251,2252],{"class":36,"line":150},[34,2253,92],{"class":47},[10,2255,2256,2257,2259,2260,2262],{},"Ce ",[14,2258,346],{}," compte : un load balancer est martele par de nombreuses goroutines a la fois, et un simple ",[14,2261,350],{}," ferait une race et donnerait le meme index a deux appelants. Avec l'atomique, le round robin est correct, sans verrou, et a peu pres aussi bon marche qu'un dispatch peut l'etre.",[10,2264,2265,2266,2269,2270,2273,2274,2277],{},"Le round robin fait une promesse et la tient : au fil du temps, chaque backend recoit le meme ",[356,2267,2268],{},"nombre"," de requetes. L'ennui est qu'egal n'est pas la meme chose qu'equitable. Le round robin suppose que chaque serveur est identique et que chaque requete coute pareil, et les deux hypotheses sont d'habitude fausses. Envoyez le meme nombre de requetes a une machine 16 coeurs et a une 2 coeurs et vous en avez surcharge une pendant que l'autre tourne au ralenti. Cet ecart entre ",[356,2271,2272],{},"egal"," et ",[356,2275,2276],{},"equitable"," est le defaut que l'algorithme suivant existe pour reparer.",[163,2279,2281],{"id":2280},"round-robin-pondere-quand-les-serveurs-ne-sont-pas-egaux","Round robin pondere : quand les serveurs ne sont pas egaux",[10,2283,2284],{},"Si un backend peut encaisser trois fois la charge d'un autre, il devrait recevoir trois fois le trafic. Attachez un poids a chaque serveur et repartissez en proportion.",[10,2286,2287,2288,2290],{},"L'implementation evidente est d'etendre la liste, en repetant chaque backend autant de fois que son poids, et de faire du round robin dessus. Ca marche, mais ca cluster : un serveur de poids 3 recoit trois requetes en rafale, puis rien, ce qui pique la charge au lieu de la lisser. La version qu'utilisent les vrais proxies est le ",[379,2289,381],{}," (c'est l'algorithme de Nginx), qui entrelace les choix pour qu'un serveur de poids 3 soit reparti uniformement dans la rotation plutot qu'entasse.",[25,2292,2294],{"className":27,"code":2293,"language":29,"meta":30,"style":30},"type SmoothWRR struct {\n    backends []*Backend\n    current  []int \u002F\u002F \"credit\" courant par backend\n    mu       sync.Mutex\n}\n\nfunc (w *SmoothWRR) Next(_ *Request) *Backend {\n    w.mu.Lock()\n    defer w.mu.Unlock()\n\n    total := 0\n    best := -1\n    for i, b := range w.backends {\n        w.current[i] += b.Weight \u002F\u002F chaque backend gagne son poids en credit\n        total += b.Weight\n        if best == -1 || w.current[i] > w.current[best] {\n            best = i \u002F\u002F le backend le plus credite gagne ce tour\n        }\n    }\n    w.current[best] -= total \u002F\u002F et rembourse\n    return w.backends[best]\n}\n",[14,2295,2296,2306,2314,2325,2335,2339,2343,2375,2389,2405,2409,2417,2427,2449,2474,2486,2528,2539,2543,2547,2568,2584],{"__ignoreMap":30},[34,2297,2298,2300,2302,2304],{"class":36,"line":37},[34,2299,48],{"class":47},[34,2301,394],{"class":51},[34,2303,110],{"class":47},[34,2305,58],{"class":47},[34,2307,2308,2310,2312],{"class":36,"line":44},[34,2309,190],{"class":118},[34,2311,193],{"class":47},[34,2313,86],{"class":51},[34,2315,2316,2318,2320,2322],{"class":36,"line":61},[34,2317,411],{"class":118},[34,2319,414],{"class":47},[34,2321,132],{"class":122},[34,2323,2324],{"class":40}," \u002F\u002F \"credit\" courant par backend\n",[34,2326,2327,2329,2331,2333],{"class":36,"line":89},[34,2328,424],{"class":118},[34,2330,427],{"class":51},[34,2332,270],{"class":47},[34,2334,432],{"class":51},[34,2336,2337],{"class":36,"line":95},[34,2338,92],{"class":47},[34,2340,2341],{"class":36,"line":102},[34,2342,99],{"emptyLinePlaceholder":98},[34,2344,2345,2347,2349,2351,2353,2355,2357,2359,2361,2363,2365,2367,2369,2371,2373],{"class":36,"line":115},[34,2346,216],{"class":47},[34,2348,219],{"class":47},[34,2350,449],{"class":71},[34,2352,225],{"class":47},[34,2354,454],{"class":51},[34,2356,81],{"class":47},[34,2358,233],{"class":64},[34,2360,68],{"class":47},[34,2362,238],{"class":71},[34,2364,75],{"class":47},[34,2366,78],{"class":51},[34,2368,81],{"class":47},[34,2370,75],{"class":47},[34,2372,249],{"class":51},[34,2374,58],{"class":47},[34,2376,2377,2379,2381,2383,2385,2387],{"class":36,"line":126},[34,2378,479],{"class":118},[34,2380,270],{"class":47},[34,2382,484],{"class":118},[34,2384,270],{"class":47},[34,2386,489],{"class":64},[34,2388,492],{"class":47},[34,2390,2391,2393,2395,2397,2399,2401,2403],{"class":36,"line":138},[34,2392,497],{"class":299},[34,2394,500],{"class":118},[34,2396,270],{"class":47},[34,2398,484],{"class":118},[34,2400,270],{"class":47},[34,2402,509],{"class":64},[34,2404,492],{"class":47},[34,2406,2407],{"class":36,"line":150},[34,2408,99],{"emptyLinePlaceholder":98},[34,2410,2411,2413,2415],{"class":36,"line":159},[34,2412,520],{"class":118},[34,2414,264],{"class":47},[34,2416,525],{"class":290},[34,2418,2419,2421,2423,2425],{"class":36,"line":528},[34,2420,531],{"class":118},[34,2422,264],{"class":47},[34,2424,536],{"class":47},[34,2426,539],{"class":290},[34,2428,2429,2431,2433,2435,2437,2439,2441,2443,2445,2447],{"class":36,"line":542},[34,2430,545],{"class":299},[34,2432,548],{"class":118},[34,2434,287],{"class":47},[34,2436,553],{"class":118},[34,2438,264],{"class":47},[34,2440,558],{"class":299},[34,2442,500],{"class":118},[34,2444,270],{"class":47},[34,2446,565],{"class":118},[34,2448,568],{"class":47},[34,2450,2451,2453,2455,2457,2459,2461,2463,2465,2467,2469,2471],{"class":36,"line":571},[34,2452,574],{"class":118},[34,2454,270],{"class":47},[34,2456,579],{"class":118},[34,2458,311],{"class":47},[34,2460,584],{"class":118},[34,2462,587],{"class":47},[34,2464,590],{"class":47},[34,2466,593],{"class":118},[34,2468,270],{"class":47},[34,2470,598],{"class":118},[34,2472,2473],{"class":40},"\u002F\u002F chaque backend gagne son poids en credit\n",[34,2475,2476,2478,2480,2482,2484],{"class":36,"line":604},[34,2477,607],{"class":118},[34,2479,610],{"class":47},[34,2481,593],{"class":118},[34,2483,270],{"class":47},[34,2485,617],{"class":118},[34,2487,2488,2490,2492,2494,2496,2498,2500,2502,2504,2506,2508,2510,2512,2514,2516,2518,2520,2522,2524,2526],{"class":36,"line":620},[34,2489,623],{"class":299},[34,2491,626],{"class":118},[34,2493,629],{"class":47},[34,2495,536],{"class":47},[34,2497,634],{"class":290},[34,2499,637],{"class":47},[34,2501,500],{"class":118},[34,2503,270],{"class":47},[34,2505,579],{"class":118},[34,2507,311],{"class":47},[34,2509,584],{"class":118},[34,2511,587],{"class":47},[34,2513,652],{"class":47},[34,2515,500],{"class":118},[34,2517,270],{"class":47},[34,2519,579],{"class":118},[34,2521,311],{"class":47},[34,2523,663],{"class":118},[34,2525,587],{"class":47},[34,2527,58],{"class":47},[34,2529,2530,2532,2534,2536],{"class":36,"line":670},[34,2531,673],{"class":118},[34,2533,676],{"class":47},[34,2535,679],{"class":118},[34,2537,2538],{"class":40},"\u002F\u002F le backend le plus credite gagne ce tour\n",[34,2540,2541],{"class":36,"line":685},[34,2542,688],{"class":47},[34,2544,2545],{"class":36,"line":691},[34,2546,694],{"class":47},[34,2548,2549,2551,2553,2555,2557,2559,2561,2563,2565],{"class":36,"line":697},[34,2550,479],{"class":118},[34,2552,270],{"class":47},[34,2554,579],{"class":118},[34,2556,311],{"class":47},[34,2558,663],{"class":118},[34,2560,587],{"class":47},[34,2562,712],{"class":47},[34,2564,715],{"class":118},[34,2566,2567],{"class":40},"\u002F\u002F et rembourse\n",[34,2569,2570,2572,2574,2576,2578,2580,2582],{"class":36,"line":721},[34,2571,300],{"class":299},[34,2573,500],{"class":118},[34,2575,270],{"class":47},[34,2577,308],{"class":118},[34,2579,311],{"class":47},[34,2581,663],{"class":118},[34,2583,736],{"class":47},[34,2585,2586],{"class":36,"line":739},[34,2587,92],{"class":47},[10,2589,2590],{},"Chaque backend accumule un credit egal a son poids a chaque tour ; celui qui a le plus de credit est choisi puis se voit facturer le poids total en retour. Sur un cycle complet un serveur de poids 3 gagne trois fois plus souvent, mais les gains sont repartis, pas entasses. Les pics ont disparu.",[10,2592,2593,2594,2597],{},"Le round robin pondere repare les differences de capacite statiques. Ce qu'il ne peut pas voir, c'est la charge ",[356,2595,2596],{},"dynamique",". Les poids sont configures a l'avance et ne changent jamais, donc si un \"petit\" serveur attrape par hasard une rafale de requetes lentes, le round robin pondere continue de le nourrir a son rythme configure, aveugle au fait qu'il se noie a l'instant present. Les poids decrivent la capacite en theorie, pas la charge dans l'instant.",[163,2599,2601],{"id":2600},"least-connections-reagir-a-la-charge-reelle","Least connections : reagir a la charge reelle",[10,2603,2604],{},"Pour repondre a ce qui se passe vraiment, cessez de compter les tours et commencez a compter le travail en cours. Least connections route chaque requete vers le backend qui a le moins de connexions actives, donc un serveur englue par des requetes lentes cesse naturellement d'en recevoir de nouvelles le temps de rattraper.",[25,2606,2608],{"className":27,"code":2607,"language":29,"meta":30,"style":30},"func (lc *LeastConn) Next(_ *Request) *Backend {\n    var best *Backend\n    min := int64(math.MaxInt64)\n    for _, b := range lc.backends {\n        c := atomic.LoadInt64(&b.ActiveConns)\n        if c \u003C min {\n            min, best = c, b\n        }\n    }\n    return best \u002F\u002F l'appelant incremente ActiveConns, decremente a la fin de la requete\n}\n",[14,2609,2610,2642,2652,2670,2692,2714,2726,2742,2746,2750,2759],{"__ignoreMap":30},[34,2611,2612,2614,2616,2618,2620,2622,2624,2626,2628,2630,2632,2634,2636,2638,2640],{"class":36,"line":37},[34,2613,216],{"class":47},[34,2615,219],{"class":47},[34,2617,772],{"class":71},[34,2619,225],{"class":47},[34,2621,777],{"class":51},[34,2623,81],{"class":47},[34,2625,233],{"class":64},[34,2627,68],{"class":47},[34,2629,238],{"class":71},[34,2631,75],{"class":47},[34,2633,78],{"class":51},[34,2635,81],{"class":47},[34,2637,75],{"class":47},[34,2639,249],{"class":51},[34,2641,58],{"class":47},[34,2643,2644,2646,2648,2650],{"class":36,"line":44},[34,2645,802],{"class":47},[34,2647,626],{"class":118},[34,2649,225],{"class":47},[34,2651,86],{"class":51},[34,2653,2654,2656,2658,2660,2662,2664,2666,2668],{"class":36,"line":61},[34,2655,813],{"class":118},[34,2657,264],{"class":47},[34,2659,818],{"class":122},[34,2661,68],{"class":47},[34,2663,823],{"class":118},[34,2665,270],{"class":47},[34,2667,828],{"class":118},[34,2669,294],{"class":47},[34,2671,2672,2674,2676,2678,2680,2682,2684,2686,2688,2690],{"class":36,"line":89},[34,2673,545],{"class":299},[34,2675,837],{"class":118},[34,2677,287],{"class":47},[34,2679,553],{"class":118},[34,2681,264],{"class":47},[34,2683,558],{"class":299},[34,2685,848],{"class":118},[34,2687,270],{"class":47},[34,2689,565],{"class":118},[34,2691,568],{"class":47},[34,2693,2694,2696,2698,2700,2702,2704,2706,2708,2710,2712],{"class":36,"line":95},[34,2695,859],{"class":118},[34,2697,264],{"class":47},[34,2699,267],{"class":118},[34,2701,270],{"class":47},[34,2703,868],{"class":64},[34,2705,276],{"class":47},[34,2707,873],{"class":118},[34,2709,270],{"class":47},[34,2711,878],{"class":118},[34,2713,294],{"class":47},[34,2715,2716,2718,2720,2722,2724],{"class":36,"line":102},[34,2717,623],{"class":299},[34,2719,887],{"class":118},[34,2721,890],{"class":47},[34,2723,893],{"class":118},[34,2725,568],{"class":47},[34,2727,2728,2730,2732,2734,2736,2738,2740],{"class":36,"line":115},[34,2729,900],{"class":118},[34,2731,287],{"class":47},[34,2733,626],{"class":118},[34,2735,676],{"class":47},[34,2737,909],{"class":118},[34,2739,287],{"class":47},[34,2741,914],{"class":118},[34,2743,2744],{"class":36,"line":126},[34,2745,688],{"class":47},[34,2747,2748],{"class":36,"line":138},[34,2749,694],{"class":47},[34,2751,2752,2754,2756],{"class":36,"line":150},[34,2753,300],{"class":299},[34,2755,626],{"class":118},[34,2757,2758],{"class":40},"\u002F\u002F l'appelant incremente ActiveConns, decremente a la fin de la requete\n",[34,2760,2761],{"class":36,"line":159},[34,2762,92],{"class":47},[10,2764,2765],{},"Cela s'adapte a la realite d'une facon que le round robin ne pourra jamais. Des durees de requete inegales, un backend qui passe une mauvaise minute, un downstream lent : least connections contourne tout cela automatiquement, car un serveur en difficulte accumule des connexions et tombe en fin de file. C'est le premier algorithme ici qui reagit a la charge au lieu de la supposer.",[10,2767,2768,2769,2772,2773,2775],{},"Le cout est dans cette boucle. Pour choisir le minimum, least connections scanne ",[356,2770,2771],{},"chaque"," backend a ",[356,2774,2771],{}," requete, ce qui va pour une poignee de serveurs et fait mal pour des centaines. Pire, sous forte concurrence, le lire-puis-router fait une race : de nombreuses goroutines observent toutes le meme backend le moins charge au meme instant et le stampedent ensemble, precisement le desequilibre que l'algorithme devait empecher. Lire un etat global pour prendre une decision locale ne passe pas a l'echelle.",[163,2777,2779],{"id":2778},"power-of-two-choices-lastuce-que-personne-nenseigne","Power of two choices : l'astuce que personne n'enseigne",[10,2781,2782,2783,2786],{},"Voici l'algorithme qui devrait etre bien plus celebre qu'il ne l'est. Au lieu de scanner tous les backends pour le vrai minimum, choisissez ",[379,2784,2785],{},"deux backends au hasard"," et routez vers le moins charge des deux. C'est toute l'idee.",[25,2788,2790],{"className":27,"code":2789,"language":29,"meta":30,"style":30},"func (p *P2C) Next(_ *Request) *Backend {\n    a := p.backends[rand.Intn(len(p.backends))]\n    b := p.backends[rand.Intn(len(p.backends))]\n    \u002F\u002F comparer seulement ces deux-la, pas toute la flotte\n    if atomic.LoadInt64(&a.ActiveConns) \u003C= atomic.LoadInt64(&b.ActiveConns) {\n        return a\n    }\n    return b\n}\n",[14,2791,2792,2824,2858,2892,2897,2937,2943,2947,2953],{"__ignoreMap":30},[34,2793,2794,2796,2798,2800,2802,2804,2806,2808,2810,2812,2814,2816,2818,2820,2822],{"class":36,"line":37},[34,2795,216],{"class":47},[34,2797,219],{"class":47},[34,2799,973],{"class":71},[34,2801,225],{"class":47},[34,2803,978],{"class":51},[34,2805,81],{"class":47},[34,2807,233],{"class":64},[34,2809,68],{"class":47},[34,2811,238],{"class":71},[34,2813,75],{"class":47},[34,2815,78],{"class":51},[34,2817,81],{"class":47},[34,2819,75],{"class":47},[34,2821,249],{"class":51},[34,2823,58],{"class":47},[34,2825,2826,2828,2830,2832,2834,2836,2838,2840,2842,2844,2846,2848,2850,2852,2854,2856],{"class":36,"line":44},[34,2827,1003],{"class":118},[34,2829,264],{"class":47},[34,2831,1008],{"class":118},[34,2833,270],{"class":47},[34,2835,308],{"class":118},[34,2837,311],{"class":47},[34,2839,1017],{"class":118},[34,2841,270],{"class":47},[34,2843,1022],{"class":64},[34,2845,68],{"class":47},[34,2847,325],{"class":64},[34,2849,68],{"class":47},[34,2851,10],{"class":118},[34,2853,270],{"class":47},[34,2855,308],{"class":118},[34,2857,336],{"class":47},[34,2859,2860,2862,2864,2866,2868,2870,2872,2874,2876,2878,2880,2882,2884,2886,2888,2890],{"class":36,"line":61},[34,2861,1041],{"class":118},[34,2863,264],{"class":47},[34,2865,1008],{"class":118},[34,2867,270],{"class":47},[34,2869,308],{"class":118},[34,2871,311],{"class":47},[34,2873,1017],{"class":118},[34,2875,270],{"class":47},[34,2877,1022],{"class":64},[34,2879,68],{"class":47},[34,2881,325],{"class":64},[34,2883,68],{"class":47},[34,2885,10],{"class":118},[34,2887,270],{"class":47},[34,2889,308],{"class":118},[34,2891,336],{"class":47},[34,2893,2894],{"class":36,"line":89},[34,2895,2896],{"class":40},"    \u002F\u002F comparer seulement ces deux-la, pas toute la flotte\n",[34,2898,2899,2901,2903,2905,2907,2909,2911,2913,2915,2917,2919,2921,2923,2925,2927,2929,2931,2933,2935],{"class":36,"line":95},[34,2900,1081],{"class":299},[34,2902,267],{"class":118},[34,2904,270],{"class":47},[34,2906,868],{"class":64},[34,2908,276],{"class":47},[34,2910,1092],{"class":118},[34,2912,270],{"class":47},[34,2914,878],{"class":118},[34,2916,81],{"class":47},[34,2918,1101],{"class":47},[34,2920,267],{"class":118},[34,2922,270],{"class":47},[34,2924,868],{"class":64},[34,2926,276],{"class":47},[34,2928,873],{"class":118},[34,2930,270],{"class":47},[34,2932,878],{"class":118},[34,2934,81],{"class":47},[34,2936,58],{"class":47},[34,2938,2939,2941],{"class":36,"line":102},[34,2940,1124],{"class":299},[34,2942,1127],{"class":118},[34,2944,2945],{"class":36,"line":115},[34,2946,694],{"class":47},[34,2948,2949,2951],{"class":36,"line":126},[34,2950,300],{"class":299},[34,2952,914],{"class":118},[34,2954,2955],{"class":36,"line":138},[34,2956,92],{"class":47},[10,2958,2959,2960,2963],{},"Ca a l'air trop simple pour etre bon, et le resultat est vraiment surprenant : n'echantillonner que deux et prendre le meilleur donne une repartition de charge presque aussi egale que scanner toute la flotte, a un cout O(1) au lieu de O(N). Le resultat mathematique derriere (le theoreme \"power of two choices\") montre que la charge maximale entre serveurs chute exponentiellement par rapport au pur aleatoire, et qu'un seul echantillon de plus achete presque tout ce benefice. Et comme deux goroutines echantillonnent rarement la ",[356,2961,2962],{},"meme"," paire, le probleme de troupeau qui plombait least connections s'evapore en grande partie.",[10,2965,2966],{},"Ce n'est pas un jouet. Power of two choices est ce vers quoi les systemes de production se tournent vraiment : c'est le defaut dans les versions recentes de HAProxy, dans le Finagle de Twitter, dans le load balancing d'Envoy. C'est le point d'equilibre de toute la progression, une qualite quasi globale a cout local, et la plupart des tutos passent droit devant. Si vous retenez un algorithme de cet article, retenez celui-la.",[163,2968,2970],{"id":2969},"ip-hash-quand-le-client-doit-rester-colle","IP hash : quand le client doit rester colle",[10,2972,2973],{},"Tout ce qui precede suppose que n'importe quel backend peut servir n'importe quelle requete. Parfois c'est faux : un client a une session, un cache, ou un upload en cours lie a un serveur precis, et il doit y revenir. La reponse classique est de hasher l'IP du client et de l'utiliser pour choisir un backend de facon deterministe.",[25,2975,2976],{"className":27,"code":1161,"language":29,"meta":30,"style":30},[14,2977,2978,3010,3024,3046,3082],{"__ignoreMap":30},[34,2979,2980,2982,2984,2986,2988,2990,2992,2994,2996,2998,3000,3002,3004,3006,3008],{"class":36,"line":37},[34,2981,216],{"class":47},[34,2983,219],{"class":47},[34,2985,1172],{"class":71},[34,2987,225],{"class":47},[34,2989,1177],{"class":51},[34,2991,81],{"class":47},[34,2993,233],{"class":64},[34,2995,68],{"class":47},[34,2997,72],{"class":71},[34,2999,75],{"class":47},[34,3001,78],{"class":51},[34,3003,81],{"class":47},[34,3005,75],{"class":47},[34,3007,249],{"class":51},[34,3009,58],{"class":47},[34,3011,3012,3014,3016,3018,3020,3022],{"class":36,"line":44},[34,3013,1202],{"class":118},[34,3015,264],{"class":47},[34,3017,1207],{"class":118},[34,3019,270],{"class":47},[34,3021,1212],{"class":64},[34,3023,492],{"class":47},[34,3025,3026,3028,3030,3032,3034,3036,3038,3040,3042,3044],{"class":36,"line":61},[34,3027,1219],{"class":118},[34,3029,270],{"class":47},[34,3031,1224],{"class":64},[34,3033,1227],{"class":47},[34,3035,1230],{"class":122},[34,3037,68],{"class":47},[34,3039,72],{"class":118},[34,3041,270],{"class":47},[34,3043,1239],{"class":118},[34,3045,1242],{"class":47},[34,3047,3048,3050,3052,3054,3056,3058,3060,3062,3064,3066,3068,3070,3072,3074,3076,3078,3080],{"class":36,"line":89},[34,3049,300],{"class":299},[34,3051,1249],{"class":118},[34,3053,270],{"class":47},[34,3055,308],{"class":118},[34,3057,311],{"class":47},[34,3059,1258],{"class":118},[34,3061,270],{"class":47},[34,3063,1263],{"class":64},[34,3065,1266],{"class":47},[34,3067,1269],{"class":122},[34,3069,68],{"class":47},[34,3071,325],{"class":64},[34,3073,68],{"class":47},[34,3075,1278],{"class":118},[34,3077,270],{"class":47},[34,3079,308],{"class":118},[34,3081,336],{"class":47},[34,3083,3084],{"class":36,"line":95},[34,3085,92],{"class":47},[10,3087,3088,3089,3091,3092,3094,3095,3098],{},"Meme IP, meme hash, meme backend, a chaque fois. Les sessions restent epinglees. C'est propre et ca marche, et ca cache un piege qui ne se declenche que lorsque vous changez le pool. Ce ",[14,3090,1294],{}," est tout le probleme : des l'instant ou vous ajoutez ou retirez un seul serveur, ",[14,3093,1298],{}," change, et ",[356,3096,3097],{},"presque tous"," les clients hashent soudain vers un backend different. Chaque session casse, chaque cache devient froid, d'un coup, pour un changement d'un seul serveur. Le hachage naif est stable seulement tant que le pool ne change jamais, ce qui n'est pas une propriete des vrais systemes.",[163,3100,3102],{"id":3101},"consistent-hashing-la-stabilite-dans-le-changement","Consistent hashing : la stabilite dans le changement",[10,3104,3105],{},"La solution est de hasher serveurs et clients sur un meme anneau abstrait, un espace de, disons, 0 a 2^32, et de marcher dans le sens horaire depuis la position du client jusqu'au premier serveur sur l'anneau. Ajoutez ou retirez un serveur et seules les cles dans l'arc de ce serveur bougent ; tous les autres restent en place.",[25,3107,3109],{"className":27,"code":3108,"language":29,"meta":30,"style":30},"type Ring struct {\n    sorted []uint32          \u002F\u002F positions de hash, triees\n    owner  map[uint32]*Backend\n}\n\nfunc (r *Ring) Next(req *Request) *Backend {\n    h := hash(req.ClientIP)\n    \u002F\u002F recherche binaire de la premiere position >= h, avec wrap\n    i := sort.Search(len(r.sorted), func(i int) bool { return r.sorted[i] >= h })\n    if i == len(r.sorted) {\n        i = 0 \u002F\u002F au-dela de la fin de l'anneau, on repart au debut\n    }\n    return r.owner[r.sorted[i]]\n}\n",[14,3110,3111,3121,3132,3144,3148,3152,3184,3202,3207,3265,3287,3298,3302,3326],{"__ignoreMap":30},[34,3112,3113,3115,3117,3119],{"class":36,"line":37},[34,3114,48],{"class":47},[34,3116,1322],{"class":51},[34,3118,110],{"class":47},[34,3120,58],{"class":47},[34,3122,3123,3125,3127,3129],{"class":36,"line":44},[34,3124,1331],{"class":118},[34,3126,414],{"class":47},[34,3128,1269],{"class":122},[34,3130,3131],{"class":40},"          \u002F\u002F positions de hash, triees\n",[34,3133,3134,3136,3138,3140,3142],{"class":36,"line":61},[34,3135,1343],{"class":118},[34,3137,1346],{"class":47},[34,3139,1269],{"class":122},[34,3141,1351],{"class":47},[34,3143,86],{"class":51},[34,3145,3146],{"class":36,"line":89},[34,3147,92],{"class":47},[34,3149,3150],{"class":36,"line":95},[34,3151,99],{"emptyLinePlaceholder":98},[34,3153,3154,3156,3158,3160,3162,3164,3166,3168,3170,3172,3174,3176,3178,3180,3182],{"class":36,"line":102},[34,3155,216],{"class":47},[34,3157,219],{"class":47},[34,3159,1370],{"class":71},[34,3161,225],{"class":47},[34,3163,1375],{"class":51},[34,3165,81],{"class":47},[34,3167,233],{"class":64},[34,3169,68],{"class":47},[34,3171,1384],{"class":71},[34,3173,75],{"class":47},[34,3175,78],{"class":51},[34,3177,81],{"class":47},[34,3179,75],{"class":47},[34,3181,249],{"class":51},[34,3183,58],{"class":47},[34,3185,3186,3188,3190,3192,3194,3196,3198,3200],{"class":36,"line":115},[34,3187,1401],{"class":118},[34,3189,264],{"class":47},[34,3191,1406],{"class":64},[34,3193,68],{"class":47},[34,3195,1384],{"class":118},[34,3197,270],{"class":47},[34,3199,1239],{"class":118},[34,3201,294],{"class":47},[34,3203,3204],{"class":36,"line":126},[34,3205,3206],{"class":40},"    \u002F\u002F recherche binaire de la premiere position >= h, avec wrap\n",[34,3208,3209,3211,3213,3215,3217,3219,3221,3223,3225,3227,3229,3231,3233,3235,3237,3239,3241,3243,3245,3247,3249,3251,3253,3255,3257,3259,3261,3263],{"class":36,"line":138},[34,3210,1426],{"class":118},[34,3212,264],{"class":47},[34,3214,1431],{"class":118},[34,3216,270],{"class":47},[34,3218,1436],{"class":64},[34,3220,68],{"class":47},[34,3222,325],{"class":64},[34,3224,68],{"class":47},[34,3226,72],{"class":118},[34,3228,270],{"class":47},[34,3230,1449],{"class":118},[34,3232,1452],{"class":47},[34,3234,1455],{"class":47},[34,3236,584],{"class":71},[34,3238,1460],{"class":122},[34,3240,81],{"class":47},[34,3242,1465],{"class":122},[34,3244,1468],{"class":47},[34,3246,1471],{"class":299},[34,3248,1474],{"class":118},[34,3250,270],{"class":47},[34,3252,1449],{"class":118},[34,3254,311],{"class":47},[34,3256,584],{"class":118},[34,3258,587],{"class":47},[34,3260,1487],{"class":47},[34,3262,1490],{"class":118},[34,3264,1493],{"class":47},[34,3266,3267,3269,3271,3273,3275,3277,3279,3281,3283,3285],{"class":36,"line":150},[34,3268,1081],{"class":299},[34,3270,679],{"class":118},[34,3272,629],{"class":47},[34,3274,1504],{"class":64},[34,3276,68],{"class":47},[34,3278,72],{"class":118},[34,3280,270],{"class":47},[34,3282,1449],{"class":118},[34,3284,81],{"class":47},[34,3286,58],{"class":47},[34,3288,3289,3291,3293,3295],{"class":36,"line":159},[34,3290,1521],{"class":118},[34,3292,676],{"class":47},[34,3294,1526],{"class":290},[34,3296,3297],{"class":40}," \u002F\u002F au-dela de la fin de l'anneau, on repart au debut\n",[34,3299,3300],{"class":36,"line":528},[34,3301,694],{"class":47},[34,3303,3304,3306,3308,3310,3312,3314,3316,3318,3320,3322,3324],{"class":36,"line":542},[34,3305,300],{"class":299},[34,3307,1474],{"class":118},[34,3309,270],{"class":47},[34,3311,1544],{"class":118},[34,3313,311],{"class":47},[34,3315,72],{"class":118},[34,3317,270],{"class":47},[34,3319,1449],{"class":118},[34,3321,311],{"class":47},[34,3323,584],{"class":118},[34,3325,1559],{"class":47},[34,3327,3328],{"class":36,"line":571},[34,3329,92],{"class":47},[10,3331,3332,3333,3336,3337,3339,3340,3343],{},"Avec le hachage naif, retirer un serveur sur N remappe a peu pres ",[356,3334,3335],{},"toutes"," les cles. Avec le consistent hashing, il en remappe seulement environ ",[14,3338,1573],{},", celles qui vivaient dans l'arc du serveur parti. En pratique on ajoute aussi des ",[379,3341,3342],{},"noeuds virtuels",", en hashant chaque backend vers de nombreuses positions sur l'anneau au lieu d'une seule, pour que la charge se repartisse uniformement et qu'aucun serveur ne possede un arc geant par accident. C'est la meme structure derriere les caches distribues et le sharding par hash : le consistent hashing est moins une astuce de load balancing qu'une reponse generale a \"assigner des cles a des noeuds pour qu'ajouter un noeud ne derange presque rien\".",[163,3345,3347],{"id":3346},"la-panne-quaucun-algorithme-ne-repare-les-serveurs-morts","La panne qu'aucun algorithme ne repare : les serveurs morts",[10,3349,3350,3351,3354],{},"Prenez du recul et remarquez ce que chaque algorithme ci-dessus suppose discretement : que le backend qu'il choisit est ",[356,3352,3353],{},"vivant",". Le round robin routera consciencieusement un tiers de votre trafic vers un serveur mort il y a trente secondes. Power of two choices echantillonnera un cadavre. Le consistent hashing epinglera un client a un backend qui a cesse de repondre. Un algorithme de selection impeccable pointe vers un serveur mort renvoie des erreurs aussi fidelement qu'il renverrait des succes.",[10,3356,3357],{},"Un vrai load balancer a donc besoin d'une couche qu'aucun de ces algorithmes ne fournit : savoir quels backends sont reellement debout. Deux approches, d'habitude combinees :",[1594,3359,3360,3369],{},[1597,3361,3362,3365,3366,3368],{},[379,3363,3364],{},"Health checks actifs."," Sonder periodiquement chaque backend (un ",[14,3367,1605],{}," sur un timer). Le marquer malsain en cas d'echec, le retirer du pool, et continuer a le sonder pour qu'il puisse revenir en se retablissant.",[1597,3370,3371,3374],{},[379,3372,3373],{},"Health checks passifs."," Observer le trafic reel. Si un backend commence a renvoyer des erreurs ou a timeouter, l'ejecter sans attendre la prochaine sonde. C'est la version load balancer d'un circuit breaker.",[25,3376,3378],{"className":27,"code":3377,"language":29,"meta":30,"style":30},"func (rr *RoundRobin) Next(_ *Request) *Backend {\n    for i := 0; i \u003C len(rr.backends); i++ {\n        n := atomic.AddUint64(&rr.counter, 1)\n        b := rr.backends[n%uint64(len(rr.backends))]\n        if b.Healthy { \u002F\u002F sauter les backends ejectes, essayer le slot suivant\n            return b\n        }\n    }\n    return nil \u002F\u002F tout le pool est down ; l'appelant doit shed ou echouer vite\n}\n",[14,3379,3380,3412,3446,3472,3506,3521,3527,3531,3535,3544],{"__ignoreMap":30},[34,3381,3382,3384,3386,3388,3390,3392,3394,3396,3398,3400,3402,3404,3406,3408,3410],{"class":36,"line":37},[34,3383,216],{"class":47},[34,3385,219],{"class":47},[34,3387,222],{"class":71},[34,3389,225],{"class":47},[34,3391,228],{"class":51},[34,3393,81],{"class":47},[34,3395,233],{"class":64},[34,3397,68],{"class":47},[34,3399,238],{"class":71},[34,3401,75],{"class":47},[34,3403,78],{"class":51},[34,3405,81],{"class":47},[34,3407,75],{"class":47},[34,3409,249],{"class":51},[34,3411,58],{"class":47},[34,3413,3414,3416,3418,3420,3422,3424,3426,3428,3430,3432,3434,3436,3438,3440,3442,3444],{"class":36,"line":44},[34,3415,545],{"class":299},[34,3417,679],{"class":118},[34,3419,264],{"class":47},[34,3421,1526],{"class":290},[34,3423,1662],{"class":47},[34,3425,679],{"class":118},[34,3427,890],{"class":47},[34,3429,1504],{"class":64},[34,3431,68],{"class":47},[34,3433,279],{"class":118},[34,3435,270],{"class":47},[34,3437,308],{"class":118},[34,3439,1679],{"class":47},[34,3441,548],{"class":118},[34,3443,1684],{"class":47},[34,3445,58],{"class":47},[34,3447,3448,3450,3452,3454,3456,3458,3460,3462,3464,3466,3468,3470],{"class":36,"line":61},[34,3449,1691],{"class":118},[34,3451,264],{"class":47},[34,3453,267],{"class":118},[34,3455,270],{"class":47},[34,3457,273],{"class":64},[34,3459,276],{"class":47},[34,3461,279],{"class":118},[34,3463,270],{"class":47},[34,3465,284],{"class":118},[34,3467,287],{"class":47},[34,3469,291],{"class":290},[34,3471,294],{"class":47},[34,3473,3474,3476,3478,3480,3482,3484,3486,3488,3490,3492,3494,3496,3498,3500,3502,3504],{"class":36,"line":89},[34,3475,1718],{"class":118},[34,3477,264],{"class":47},[34,3479,303],{"class":118},[34,3481,270],{"class":47},[34,3483,308],{"class":118},[34,3485,311],{"class":47},[34,3487,314],{"class":118},[34,3489,317],{"class":47},[34,3491,320],{"class":122},[34,3493,68],{"class":47},[34,3495,325],{"class":64},[34,3497,68],{"class":47},[34,3499,279],{"class":118},[34,3501,270],{"class":47},[34,3503,308],{"class":118},[34,3505,336],{"class":47},[34,3507,3508,3510,3512,3514,3516,3518],{"class":36,"line":95},[34,3509,623],{"class":299},[34,3511,593],{"class":118},[34,3513,270],{"class":47},[34,3515,1759],{"class":118},[34,3517,1762],{"class":47},[34,3519,3520],{"class":40}," \u002F\u002F sauter les backends ejectes, essayer le slot suivant\n",[34,3522,3523,3525],{"class":36,"line":102},[34,3524,1770],{"class":299},[34,3526,914],{"class":118},[34,3528,3529],{"class":36,"line":115},[34,3530,688],{"class":47},[34,3532,3533],{"class":36,"line":126},[34,3534,694],{"class":47},[34,3536,3537,3539,3541],{"class":36,"line":138},[34,3538,300],{"class":299},[34,3540,1787],{"class":47},[34,3542,3543],{"class":40}," \u002F\u002F tout le pool est down ; l'appelant doit shed ou echouer vite\n",[34,3545,3546],{"class":36,"line":150},[34,3547,92],{"class":47},[10,3549,3550,3551,3554,3555,3558,3559,3562,3563,3566,3567,3570],{},"Cela boucle avec le reste de la boite a outils de resilience. L'ejection par health check est le cousin cote selection-de-serveur du ",[1092,3552,3553],{"href":1800},"circuit breaker qu'un client utilise pour cesser d'appeler une dependance morte",", et il se tient a cote du ",[1092,3556,3557],{"href":1805},"rate limiting qui deleste la charge qu'un service ne peut servir",". Le load balancing est la troisieme face de la meme piece : le rate limiting decide ",[356,3560,3561],{},"combien"," un serveur prend, les retries decident ",[356,3564,3565],{},"quand"," un client envoie, et le load balancing decide ",[356,3568,3569],{},"quel"," serveur le recoit. Rendez la selection parfaite et sautez les health checks, et vous avez construit une facon tres equitable de repartir la panne.",[163,3572,3574],{"id":3573},"la-chaine-en-un-coup-doeil","La chaine, en un coup d'oeil",[1825,3576,3577,3590],{},[1828,3578,3579],{},[1831,3580,3581,3584,3587],{},[1834,3582,3583],{},"Algorithme",[1834,3585,3586],{},"Le defaut qu'il repare",[1834,3588,3589],{},"Ce qu'il ne peut toujours pas faire",[1844,3591,3592,3602,3613,3623,3633,3643,3653],{},[1831,3593,3594,3596,3599],{},[1849,3595,1851],{},[1849,3597,3598],{},"Le clustering de l'aleatoire ; tours strictement egaux",[1849,3600,3601],{},"Traite serveurs et requetes inegaux comme egaux",[1831,3603,3604,3607,3610],{},[1849,3605,3606],{},"Round robin pondere",[1849,3608,3609],{},"Serveurs de capacite differente",[1849,3611,3612],{},"Poids statiques, aveugles a la charge en direct",[1831,3614,3615,3617,3620],{},[1849,3616,1873],{},[1849,3618,3619],{},"Reagit a la charge reelle en cours",[1849,3621,3622],{},"Scan O(N) par requete ; troupeau sous concurrence",[1831,3624,3625,3627,3630],{},[1849,3626,1884],{},[1849,3628,3629],{},"Le cout et le troupeau de least-connections",[1849,3631,3632],{},"Suppose que tout backend peut servir toute requete",[1831,3634,3635,3637,3640],{},[1849,3636,1895],{},[1849,3638,3639],{},"La stickiness de session vers un backend",[1849,3641,3642],{},"Rehashe tout quand le pool change",[1831,3644,3645,3647,3650],{},[1849,3646,1906],{},[1849,3648,3649],{},"La stabilite quand les serveurs vont et viennent",[1849,3651,3652],{},"Rien, si vous routez vers des backends morts",[1831,3654,3655,3657,3660],{},[1849,3656,1917],{},[1849,3658,3659],{},"Router vers des serveurs down",[1849,3661,3662],{},"n\u002Fa : c'est la couche sous tous les autres",[10,3664,3665],{},"Aucun n'est \"le meilleur\". Chacun est la bonne reponse a une pression precise, et chacun a ete saisi parce que le precedent a heurte un mur.",[163,3667,3669],{"id":3668},"a-retenir","A retenir",[1594,3671,3672,3678,3684,3693,3699],{},[1597,3673,3674,3677],{},[379,3675,3676],{},"Commencez plus simple que vous ne le croyez necessaire."," Pour une petite flotte de serveurs similaires avec des requetes similaires, le round robin ou meme le pur aleatoire va vraiment bien. N'allez plus loin que lorsqu'un vrai defaut mord.",[1597,3679,3680,3683],{},[379,3681,3682],{},"Egal n'est pas equitable."," Utilisez des poids quand les serveurs different en capacite, et des comptes de connexions quand les requetes different en cout. Le round robin melange les deux et est aveugle aux deux.",[1597,3685,3686,3689,3690,3692],{},[379,3687,3688],{},"Power of two choices est le point d'equilibre sous-estime."," Une qualite de charge quasi globale a cout O(1), sans le scan ni le troupeau de least-connections. C'est ce qu'utilisent les vrais proxies, et c'est a un appel ",[14,3691,1017],{}," de distance.",[1597,3694,3695,3698],{},[379,3696,3697],{},"Le hachage naif casse des que le pool change."," Si vous avez besoin d'un routage colle, utilisez le consistent hashing avec noeuds virtuels, ou acceptez qu'ajouter un serveur refroidira le cache de tous.",[1597,3700,3701,3704],{},[379,3702,3703],{},"Le meilleur algorithme de selection route quand meme vers des serveurs morts."," Les health checks et l'ejection ne sont pas du vernis optionnel ; ils sont la couche qui fait reellement marcher chaque algorithme ci-dessus en production.",[10,3706,3707],{},"La lecon de toute la chaine est que le \"load balancing\" n'est pas un probleme avec une reponse. C'est une suite de pressions de plus en plus precises, capacite, charge en direct, cout, stickiness, churn, panne, et chaque algorithme classique est l'outil qui repond a exactement l'une d'elles. Savoir quelle pression vous avez reellement, c'est l'essentiel de savoir quel algorithme choisir.",[1970,3709,1972],{},{"title":30,"searchDepth":44,"depth":44,"links":3711},[3712,3713,3714,3715,3716,3717,3718,3719,3720],{"id":2113,"depth":44,"text":2114},{"id":2280,"depth":44,"text":2281},{"id":2600,"depth":44,"text":2601},{"id":2778,"depth":44,"text":2779},{"id":2969,"depth":44,"text":2970},{"id":3101,"depth":44,"text":3102},{"id":3346,"depth":44,"text":3347},{"id":3573,"depth":44,"text":3574},{"id":3668,"depth":44,"text":3669},"Repartir des requetes sur N serveurs sonne comme un one-liner : choisir un serveur, envoyer la requete. Puis un backend est plus lent, ou plus gros, ou tient une session, et le choix naif s'effondre. On parcourt les algorithmes classiques comme une chaine ou chacun existe pour reparer l'angle mort du precedent : round robin, pondere, least connections, power of two choices, et consistent hashing, en Go, jusqu'a la panne que tous les tutos oublient.",{},"\u002Fbackend\u002Fload-balancing\u002Fload-balancing-algorithms.fr",{"title":2006,"description":3721},"algorithmes-load-balancing-go","2.backend\u002F3.load-balancing\u002F1.load-balancing-algorithms.fr",[1995,1996,1997,1998],[2000,2001,2002],"1aP0VGcLiweWWOWXDdgvU5zYQ53IQHOwcGtOAWW7bdo",[3731,3734],{"title":3732,"path":3733},"Retry Storms: How Good Clients Take Down Healthy Servers","\u002Fbackend\u002Fapi-design\u002Fretry-storms",{"title":3735,"path":3736},"How a job interview led me to create Obscura - A password generator with real entropy","\u002Fsecurity\u002Fintroducing-obscura",[3738,3744,3753],{"path":3733,"title":3732,"description":3739,"date":3740,"tags":3741,"topics":3743},"A retry looks harmless: the request failed, so try again. Multiply that by every client, add one slow dependency, and retries turn into a self-inflicted DDoS. This walks from the naive retry loop to exponential backoff, jitter, retry budgets and circuit breakers, the caller-side half of resilience that pairs with rate limiting on the server.","2026-07-11",[1998,1997,1996,3742],"Retries",[2000,2002,2001],{"path":3745,"title":3746,"description":3747,"date":3748,"tags":3749,"topics":3752},"\u002Fbackend\u002Fapi-design\u002Frate-limiting-at-scale","Why Redis Rate Limiting Breaks at Scale (and What Uber Does Instead)","A token bucket in memory is trivial. Put it behind Redis and it works, until it doesn't. This walks through rate limiting from one node, to a shared Redis, to why that model collapses at millions of requests per second, and the shift Uber made: enforce locally, coordinate globally, and drop by probability.","2026-07-10",[3750,1997,1996,3751],"Rate-Limiting","Performance",[2000,2002,2001],{"path":3754,"title":3755,"description":3756,"date":3757,"tags":3758,"topics":3763},"\u002Fsecurity\u002Fauthz\u002Fzanzibar-explained","Zanzibar Demystified: How Google Answers 'Can This User Do This?'","Authorization looks like a one-line if statement until you run it ten million times a second across every product Google ships. Zanzibar is the system that made that question fast, consistent and global. This walks from the naive permission check to relationship-based access control, the tuple model, consistency with zookies, and the open-source heirs like OpenFGA you can actually run today.","2026-07-18",[3759,3760,3761,1997,3762],"Authorization","ReBAC","Security","OpenFGA",[3764,2000,2002],"authz",1784570358356]