[{"data":1,"prerenderedAt":2926},["ShallowReactive",2],{"page-\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fdebugging-429-too-many-requests-errors\u002F":3,"faq-schema-\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fdebugging-429-too-many-requests-errors\u002F":2905},{"id":4,"title":5,"body":6,"description":2895,"extension":2896,"meta":2897,"navigation":275,"path":2901,"seo":2902,"stem":2903,"__hash__":2904},"content\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fdebugging-429-too-many-requests-errors\u002Findex.md","Debugging 429 Too Many Requests Errors in Python",{"type":7,"value":8,"toc":2883},"minimark",[9,13,35,40,46,59,188,191,195,206,1033,1046,1065,1069,1072,1193,1196,1499,1564,1572,1576,1582,1713,1720,2187,2198,2202,2210,2298,2737,2749,2753,2759,2762,2766,2788,2792,2795,2799,2806,2812,2818,2824,2839,2843,2879],[10,11,5],"h1",{"id":12},"debugging-429-too-many-requests-errors-in-python",[14,15,16,17,21,22,27,28,30,31,34],"p",{},"A ",[18,19,20],"code",{},"429"," is the cheapest error a provider can send you and the most expensive one to guess at. Part of the ",[23,24,26],"a",{"href":25},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002F","Making HTTP Requests with the Requests Library"," guide. This page resolves one specific decision: when your worker starts returning ",[18,29,20],{},", do you add a retry, add a limiter, or change how the fleet is keyed? Those three fixes cost wildly different amounts of engineering time, and picking the wrong one burns a weekend and still pages you at 3am. The short version — read the headers first, honour ",[18,32,33],{},"Retry-After"," second, and only then decide whether you are fighting a per-key budget or a per-IP budget, because the fix for one does nothing for the other.",[36,37,39],"h2",{"id":38},"read-the-headers-before-you-touch-the-retry-loop","Read the headers before you touch the retry loop",[14,41,42,43,45],{},"Almost every debugging session that goes badly starts with someone wrapping the failing call in a retry and shipping it. The response already told you what to do. A well-behaved provider returns three families of signal on a ",[18,44,20],{},": a wait instruction, a budget snapshot, and a scope hint. Ignore any of them and you are guessing.",[14,47,48,50,51,54,55,58],{},[18,49,33],{}," is the wait instruction. It is either an integer number of seconds or an HTTP-date, and RFC 9110 permits both, so your parser must handle both or it will silently fall through to a default backoff on half the providers you integrate. The ",[18,52,53],{},"x-ratelimit-*"," family is the budget snapshot: limit, remaining, and reset. Treat ",[18,56,57],{},"remaining"," as advisory rather than authoritative — it is computed on one edge node and your next request may land on another — but treat it as an excellent early warning. The scope hint is the one builders skip, and it is the one that decides your architecture.",[60,61,69,70,69,74,69,78,69,85,69,93,69,102,69,108,69,113,69,117,69,121,69,125,69,129,69,133,69,140,69,145,69,149,69,155,69,161,69,166,69,169,69,173,69,177,69,180,69,184],"svg",{"viewBox":62,"role":63,"ariaLabelledBy":64,"xmlns":67,"style":68},"0 0 720 292","img",[65,66],"d429-hdr-t","d429-hdr-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;height:auto;margin:1.5rem 0;font-family:var(--font-sans);","\n  ",[71,72,73],"title",{"id":65},"Anatomy of a 429 response",[75,76,77],"desc",{"id":66},"A rate-limited HTTP response with its retry-after, limit, remaining, reset and scope headers annotated with the action each one implies.",[79,80],"rect",{"x":81,"y":81,"width":82,"height":83,"fill":84},"0","720","292","var(--c-surface)",[86,87,92],"text",{"x":88,"y":89,"fill":90,"style":91},"24","34","var(--c-text-muted)","font-size:12;font-family:var(--font-sans);","What the provider actually sent back",[79,94],{"x":88,"y":95,"width":96,"height":97,"rx":98,"fill":99,"stroke":100,"style":101},"46","312","216","10","var(--c-surface-alt)","var(--c-border)","stroke-width:2;",[86,103,107],{"x":104,"y":105,"fill":106,"style":91},"40","76","var(--c-coral)","HTTP\u002F1.1 429 Too Many Requests",[86,109,112],{"x":104,"y":110,"fill":111,"style":91},"104","var(--c-text)","retry-after: 30",[86,114,116],{"x":104,"y":115,"fill":111,"style":91},"130","x-ratelimit-limit: 600",[86,118,120],{"x":104,"y":119,"fill":111,"style":91},"156","x-ratelimit-remaining: 0",[86,122,124],{"x":104,"y":123,"fill":111,"style":91},"182","x-ratelimit-reset: 1774281600",[86,126,128],{"x":104,"y":127,"fill":111,"style":91},"208","x-ratelimit-scope: api-key",[86,130,132],{"x":104,"y":131,"fill":90,"style":91},"240","body: {\"error\": \"rate_limited\"}",[134,135],"line",{"x1":136,"y1":110,"x2":137,"y2":138,"stroke":139,"style":101},"336","396","98","var(--c-teal)",[134,141],{"x1":136,"y1":142,"x2":137,"y2":143,"stroke":144,"style":101},"143","158","var(--c-blue)",[134,146],{"x1":136,"y1":127,"x2":137,"y2":147,"stroke":148,"style":101},"218","var(--c-yellow)",[79,150],{"x":137,"y":151,"width":152,"height":153,"rx":154,"fill":84,"stroke":139,"style":101},"74","300","48","8",[86,156,160],{"x":157,"y":158,"fill":111,"style":159},"546","94","text-anchor:middle;font-size:12;font-family:var(--font-sans);","Sleep exactly 30s, then retry",[86,162,165],{"x":157,"y":163,"fill":90,"style":164},"111","text-anchor:middle;font-size:11;font-family:var(--font-sans);","never invent your own number",[79,167],{"x":137,"y":168,"width":152,"height":153,"rx":154,"fill":84,"stroke":144,"style":101},"134",[86,170,172],{"x":157,"y":171,"fill":111,"style":159},"154","Budget is 600 per window",[86,174,176],{"x":157,"y":175,"fill":90,"style":164},"171","size your bucket from this",[79,178],{"x":137,"y":179,"width":152,"height":153,"rx":154,"fill":84,"stroke":148,"style":101},"194",[86,181,183],{"x":157,"y":182,"fill":111,"style":159},"214","Counter belongs to the key",[86,185,187],{"x":157,"y":186,"fill":90,"style":164},"231","more IPs will not help you",[14,189,190],{},"The reset header is the second-most-misread value here. Some providers send an absolute Unix epoch, others send seconds-until-reset. Branch on magnitude rather than trusting the docs, and log the raw string when you cannot classify it.",[36,192,194],{"id":193},"retry-after-is-not-a-suggestion","Retry-After is not a suggestion",[14,196,197,198,202,203,205],{},"Here is the client I actually ship. It uses httpx because the async path matters once you have more than one in-flight call — the trade-offs are covered in ",[23,199,201],{"href":200},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fhttpx-vs-requests-for-async\u002F","httpx vs requests for async",". Every knob comes from the environment, and the ",[18,204,20],{}," branch does exactly one thing: it obeys the server.",[207,208,213],"pre",{"className":209,"code":210,"language":211,"meta":212,"style":212},"language-python shiki shiki-themes github-light github-dark","import asyncio\nimport os\nimport random\nfrom datetime import datetime, timezone\nfrom email.utils import parsedate_to_datetime\n\nimport httpx\n\nMAX_SLEEP = float(os.getenv(\"RATE_LIMIT_MAX_SLEEP\", \"120\"))\nMAX_ATTEMPTS = int(os.getenv(\"RATE_LIMIT_MAX_ATTEMPTS\", \"5\"))\n\n\ndef seconds_from_retry_after(raw: str | None, attempt: int) -> float:\n    \"\"\"Parse Retry-After as delta-seconds or HTTP-date, else back off.\"\"\"\n    if raw is None:\n        return min(2.0**attempt, MAX_SLEEP)\n    raw = raw.strip()\n    if raw.isdigit():\n        return min(float(raw), MAX_SLEEP)\n    try:\n        when = parsedate_to_datetime(raw)\n    except (TypeError, ValueError):\n        return min(2.0**attempt, MAX_SLEEP)\n    if when.tzinfo is None:\n        when = when.replace(tzinfo=timezone.utc)\n    delta = (when - datetime.now(timezone.utc)).total_seconds()\n    return min(max(delta, 0.0), MAX_SLEEP)\n\n\nasync def get_json(client: httpx.AsyncClient, path: str) -> dict:\n    for attempt in range(MAX_ATTEMPTS):\n        response = await client.get(path)\n        if response.status_code != 429:\n            response.raise_for_status()\n            return response.json()\n        wait = seconds_from_retry_after(response.headers.get(\"retry-after\"), attempt)\n        # Full jitter keeps a fleet from re-synchronising on the same reset tick.\n        await asyncio.sleep(wait + random.uniform(0, min(wait, 1.0)))\n    raise RuntimeError(f\"still rate limited after {MAX_ATTEMPTS} attempts: {path}\")\n\n\nasync def main() -> None:\n    headers = {\"Authorization\": f\"Bearer {os.environ['PROVIDER_API_KEY']}\"}\n    timeout = httpx.Timeout(connect=3.0, read=10.0, write=10.0, pool=5.0)\n    async with httpx.AsyncClient(\n        base_url=os.environ[\"PROVIDER_BASE_URL\"], headers=headers, timeout=timeout\n    ) as client:\n        print(await get_json(client, os.getenv(\"PROVIDER_PROBE_PATH\", \"\u002Fv1\u002Faccount\")))\n\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n","python","",[18,214,215,227,235,243,257,270,277,285,290,319,342,347,352,389,395,411,437,449,457,475,483,494,514,533,547,566,583,609,614,619,643,664,678,695,701,710,727,734,765,802,807,812,830,870,920,932,964,976,1000,1005,1010,1027],{"__ignoreMap":212},[216,217,219,223],"span",{"class":134,"line":218},1,[216,220,222],{"class":221},"szBVR","import",[216,224,226],{"class":225},"sVt8B"," asyncio\n",[216,228,230,232],{"class":134,"line":229},2,[216,231,222],{"class":221},[216,233,234],{"class":225}," os\n",[216,236,238,240],{"class":134,"line":237},3,[216,239,222],{"class":221},[216,241,242],{"class":225}," random\n",[216,244,246,249,252,254],{"class":134,"line":245},4,[216,247,248],{"class":221},"from",[216,250,251],{"class":225}," datetime ",[216,253,222],{"class":221},[216,255,256],{"class":225}," datetime, timezone\n",[216,258,260,262,265,267],{"class":134,"line":259},5,[216,261,248],{"class":221},[216,263,264],{"class":225}," email.utils ",[216,266,222],{"class":221},[216,268,269],{"class":225}," parsedate_to_datetime\n",[216,271,273],{"class":134,"line":272},6,[216,274,276],{"emptyLinePlaceholder":275},true,"\n",[216,278,280,282],{"class":134,"line":279},7,[216,281,222],{"class":221},[216,283,284],{"class":225}," httpx\n",[216,286,288],{"class":134,"line":287},8,[216,289,276],{"emptyLinePlaceholder":275},[216,291,293,297,300,303,306,310,313,316],{"class":134,"line":292},9,[216,294,296],{"class":295},"sj4cs","MAX_SLEEP",[216,298,299],{"class":221}," =",[216,301,302],{"class":295}," float",[216,304,305],{"class":225},"(os.getenv(",[216,307,309],{"class":308},"sZZnC","\"RATE_LIMIT_MAX_SLEEP\"",[216,311,312],{"class":225},", ",[216,314,315],{"class":308},"\"120\"",[216,317,318],{"class":225},"))\n",[216,320,322,325,327,330,332,335,337,340],{"class":134,"line":321},10,[216,323,324],{"class":295},"MAX_ATTEMPTS",[216,326,299],{"class":221},[216,328,329],{"class":295}," int",[216,331,305],{"class":225},[216,333,334],{"class":308},"\"RATE_LIMIT_MAX_ATTEMPTS\"",[216,336,312],{"class":225},[216,338,339],{"class":308},"\"5\"",[216,341,318],{"class":225},[216,343,345],{"class":134,"line":344},11,[216,346,276],{"emptyLinePlaceholder":275},[216,348,350],{"class":134,"line":349},12,[216,351,276],{"emptyLinePlaceholder":275},[216,353,355,358,362,365,368,371,374,377,380,383,386],{"class":134,"line":354},13,[216,356,357],{"class":221},"def",[216,359,361],{"class":360},"sScJk"," seconds_from_retry_after",[216,363,364],{"class":225},"(raw: ",[216,366,367],{"class":295},"str",[216,369,370],{"class":221}," |",[216,372,373],{"class":295}," None",[216,375,376],{"class":225},", attempt: ",[216,378,379],{"class":295},"int",[216,381,382],{"class":225},") -> ",[216,384,385],{"class":295},"float",[216,387,388],{"class":225},":\n",[216,390,392],{"class":134,"line":391},14,[216,393,394],{"class":308},"    \"\"\"Parse Retry-After as delta-seconds or HTTP-date, else back off.\"\"\"\n",[216,396,398,401,404,407,409],{"class":134,"line":397},15,[216,399,400],{"class":221},"    if",[216,402,403],{"class":225}," raw ",[216,405,406],{"class":221},"is",[216,408,373],{"class":295},[216,410,388],{"class":225},[216,412,414,417,420,423,426,429,432,434],{"class":134,"line":413},16,[216,415,416],{"class":221},"        return",[216,418,419],{"class":295}," min",[216,421,422],{"class":225},"(",[216,424,425],{"class":295},"2.0",[216,427,428],{"class":221},"**",[216,430,431],{"class":225},"attempt, ",[216,433,296],{"class":295},[216,435,436],{"class":225},")\n",[216,438,440,443,446],{"class":134,"line":439},17,[216,441,442],{"class":225},"    raw ",[216,444,445],{"class":221},"=",[216,447,448],{"class":225}," raw.strip()\n",[216,450,452,454],{"class":134,"line":451},18,[216,453,400],{"class":221},[216,455,456],{"class":225}," raw.isdigit():\n",[216,458,460,462,464,466,468,471,473],{"class":134,"line":459},19,[216,461,416],{"class":221},[216,463,419],{"class":295},[216,465,422],{"class":225},[216,467,385],{"class":295},[216,469,470],{"class":225},"(raw), ",[216,472,296],{"class":295},[216,474,436],{"class":225},[216,476,478,481],{"class":134,"line":477},20,[216,479,480],{"class":221},"    try",[216,482,388],{"class":225},[216,484,486,489,491],{"class":134,"line":485},21,[216,487,488],{"class":225},"        when ",[216,490,445],{"class":221},[216,492,493],{"class":225}," parsedate_to_datetime(raw)\n",[216,495,497,500,503,506,508,511],{"class":134,"line":496},22,[216,498,499],{"class":221},"    except",[216,501,502],{"class":225}," (",[216,504,505],{"class":295},"TypeError",[216,507,312],{"class":225},[216,509,510],{"class":295},"ValueError",[216,512,513],{"class":225},"):\n",[216,515,517,519,521,523,525,527,529,531],{"class":134,"line":516},23,[216,518,416],{"class":221},[216,520,419],{"class":295},[216,522,422],{"class":225},[216,524,425],{"class":295},[216,526,428],{"class":221},[216,528,431],{"class":225},[216,530,296],{"class":295},[216,532,436],{"class":225},[216,534,536,538,541,543,545],{"class":134,"line":535},24,[216,537,400],{"class":221},[216,539,540],{"class":225}," when.tzinfo ",[216,542,406],{"class":221},[216,544,373],{"class":295},[216,546,388],{"class":225},[216,548,550,552,554,557,561,563],{"class":134,"line":549},25,[216,551,488],{"class":225},[216,553,445],{"class":221},[216,555,556],{"class":225}," when.replace(",[216,558,560],{"class":559},"s4XuR","tzinfo",[216,562,445],{"class":221},[216,564,565],{"class":225},"timezone.utc)\n",[216,567,569,572,574,577,580],{"class":134,"line":568},26,[216,570,571],{"class":225},"    delta ",[216,573,445],{"class":221},[216,575,576],{"class":225}," (when ",[216,578,579],{"class":221},"-",[216,581,582],{"class":225}," datetime.now(timezone.utc)).total_seconds()\n",[216,584,586,589,591,593,596,599,602,605,607],{"class":134,"line":585},27,[216,587,588],{"class":221},"    return",[216,590,419],{"class":295},[216,592,422],{"class":225},[216,594,595],{"class":295},"max",[216,597,598],{"class":225},"(delta, ",[216,600,601],{"class":295},"0.0",[216,603,604],{"class":225},"), ",[216,606,296],{"class":295},[216,608,436],{"class":225},[216,610,612],{"class":134,"line":611},28,[216,613,276],{"emptyLinePlaceholder":275},[216,615,617],{"class":134,"line":616},29,[216,618,276],{"emptyLinePlaceholder":275},[216,620,622,625,628,631,634,636,638,641],{"class":134,"line":621},30,[216,623,624],{"class":221},"async",[216,626,627],{"class":221}," def",[216,629,630],{"class":360}," get_json",[216,632,633],{"class":225},"(client: httpx.AsyncClient, path: ",[216,635,367],{"class":295},[216,637,382],{"class":225},[216,639,640],{"class":295},"dict",[216,642,388],{"class":225},[216,644,646,649,652,655,658,660,662],{"class":134,"line":645},31,[216,647,648],{"class":221},"    for",[216,650,651],{"class":225}," attempt ",[216,653,654],{"class":221},"in",[216,656,657],{"class":295}," range",[216,659,422],{"class":225},[216,661,324],{"class":295},[216,663,513],{"class":225},[216,665,667,670,672,675],{"class":134,"line":666},32,[216,668,669],{"class":225},"        response ",[216,671,445],{"class":221},[216,673,674],{"class":221}," await",[216,676,677],{"class":225}," client.get(path)\n",[216,679,681,684,687,690,693],{"class":134,"line":680},33,[216,682,683],{"class":221},"        if",[216,685,686],{"class":225}," response.status_code ",[216,688,689],{"class":221},"!=",[216,691,692],{"class":295}," 429",[216,694,388],{"class":225},[216,696,698],{"class":134,"line":697},34,[216,699,700],{"class":225},"            response.raise_for_status()\n",[216,702,704,707],{"class":134,"line":703},35,[216,705,706],{"class":221},"            return",[216,708,709],{"class":225}," response.json()\n",[216,711,713,716,718,721,724],{"class":134,"line":712},36,[216,714,715],{"class":225},"        wait ",[216,717,445],{"class":221},[216,719,720],{"class":225}," seconds_from_retry_after(response.headers.get(",[216,722,723],{"class":308},"\"retry-after\"",[216,725,726],{"class":225},"), attempt)\n",[216,728,730],{"class":134,"line":729},37,[216,731,733],{"class":732},"sJ8bj","        # Full jitter keeps a fleet from re-synchronising on the same reset tick.\n",[216,735,737,740,743,746,749,751,753,756,759,762],{"class":134,"line":736},38,[216,738,739],{"class":221},"        await",[216,741,742],{"class":225}," asyncio.sleep(wait ",[216,744,745],{"class":221},"+",[216,747,748],{"class":225}," random.uniform(",[216,750,81],{"class":295},[216,752,312],{"class":225},[216,754,755],{"class":295},"min",[216,757,758],{"class":225},"(wait, ",[216,760,761],{"class":295},"1.0",[216,763,764],{"class":225},")))\n",[216,766,768,771,774,776,779,782,785,788,791,794,797,800],{"class":134,"line":767},39,[216,769,770],{"class":221},"    raise",[216,772,773],{"class":295}," RuntimeError",[216,775,422],{"class":225},[216,777,778],{"class":221},"f",[216,780,781],{"class":308},"\"still rate limited after ",[216,783,784],{"class":295},"{MAX_ATTEMPTS}",[216,786,787],{"class":308}," attempts: ",[216,789,790],{"class":295},"{",[216,792,793],{"class":225},"path",[216,795,796],{"class":295},"}",[216,798,799],{"class":308},"\"",[216,801,436],{"class":225},[216,803,805],{"class":134,"line":804},40,[216,806,276],{"emptyLinePlaceholder":275},[216,808,810],{"class":134,"line":809},41,[216,811,276],{"emptyLinePlaceholder":275},[216,813,815,817,819,822,825,828],{"class":134,"line":814},42,[216,816,624],{"class":221},[216,818,627],{"class":221},[216,820,821],{"class":360}," main",[216,823,824],{"class":225},"() -> ",[216,826,827],{"class":295},"None",[216,829,388],{"class":225},[216,831,833,836,838,841,844,847,849,852,854,857,860,863,865,867],{"class":134,"line":832},43,[216,834,835],{"class":225},"    headers ",[216,837,445],{"class":221},[216,839,840],{"class":225}," {",[216,842,843],{"class":308},"\"Authorization\"",[216,845,846],{"class":225},": ",[216,848,778],{"class":221},[216,850,851],{"class":308},"\"Bearer ",[216,853,790],{"class":295},[216,855,856],{"class":225},"os.environ[",[216,858,859],{"class":308},"'PROVIDER_API_KEY'",[216,861,862],{"class":225},"]",[216,864,796],{"class":295},[216,866,799],{"class":308},[216,868,869],{"class":225},"}\n",[216,871,873,876,878,881,884,886,889,891,894,896,899,901,904,906,908,910,913,915,918],{"class":134,"line":872},44,[216,874,875],{"class":225},"    timeout ",[216,877,445],{"class":221},[216,879,880],{"class":225}," httpx.Timeout(",[216,882,883],{"class":559},"connect",[216,885,445],{"class":221},[216,887,888],{"class":295},"3.0",[216,890,312],{"class":225},[216,892,893],{"class":559},"read",[216,895,445],{"class":221},[216,897,898],{"class":295},"10.0",[216,900,312],{"class":225},[216,902,903],{"class":559},"write",[216,905,445],{"class":221},[216,907,898],{"class":295},[216,909,312],{"class":225},[216,911,912],{"class":559},"pool",[216,914,445],{"class":221},[216,916,917],{"class":295},"5.0",[216,919,436],{"class":225},[216,921,923,926,929],{"class":134,"line":922},45,[216,924,925],{"class":221},"    async",[216,927,928],{"class":221}," with",[216,930,931],{"class":225}," httpx.AsyncClient(\n",[216,933,935,938,940,942,945,948,951,953,956,959,961],{"class":134,"line":934},46,[216,936,937],{"class":559},"        base_url",[216,939,445],{"class":221},[216,941,856],{"class":225},[216,943,944],{"class":308},"\"PROVIDER_BASE_URL\"",[216,946,947],{"class":225},"], ",[216,949,950],{"class":559},"headers",[216,952,445],{"class":221},[216,954,955],{"class":225},"headers, ",[216,957,958],{"class":559},"timeout",[216,960,445],{"class":221},[216,962,963],{"class":225},"timeout\n",[216,965,967,970,973],{"class":134,"line":966},47,[216,968,969],{"class":225},"    ) ",[216,971,972],{"class":221},"as",[216,974,975],{"class":225}," client:\n",[216,977,979,982,984,987,990,993,995,998],{"class":134,"line":978},48,[216,980,981],{"class":295},"        print",[216,983,422],{"class":225},[216,985,986],{"class":221},"await",[216,988,989],{"class":225}," get_json(client, os.getenv(",[216,991,992],{"class":308},"\"PROVIDER_PROBE_PATH\"",[216,994,312],{"class":225},[216,996,997],{"class":308},"\"\u002Fv1\u002Faccount\"",[216,999,764],{"class":225},[216,1001,1003],{"class":134,"line":1002},49,[216,1004,276],{"emptyLinePlaceholder":275},[216,1006,1008],{"class":134,"line":1007},50,[216,1009,276],{"emptyLinePlaceholder":275},[216,1011,1013,1016,1019,1022,1025],{"class":134,"line":1012},51,[216,1014,1015],{"class":221},"if",[216,1017,1018],{"class":295}," __name__",[216,1020,1021],{"class":221}," ==",[216,1023,1024],{"class":308}," \"__main__\"",[216,1026,388],{"class":225},[216,1028,1030],{"class":134,"line":1029},52,[216,1031,1032],{"class":225},"    asyncio.run(main())\n",[14,1034,1035,1036,1040,1041,1045],{},"Two details earn their keep. The jitter is added ",[1037,1038,1039],"em",{},"on top of"," the server's number rather than replacing it, so you never retry earlier than instructed while still de-synchronising a fleet that all got throttled on the same second. And the attempt cap is low: five attempts with a 120-second ceiling is roughly ten minutes of patience, which is long enough for a window reset and short enough that a broken key surfaces as a real alert. If retries are becoming a large part of your control flow, move them into a declarative policy — ",[23,1042,1044],{"href":1043},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fretrying-failed-http-requests-with-tenacity\u002F","retrying failed HTTP requests with tenacity"," covers that migration.",[14,1047,1048,1049,1051,1052,1055,1056,1059,1060,1064],{},"One warning: retry only ",[18,1050,20],{}," and ",[18,1053,1054],{},"5xx",". A ",[18,1057,1058],{},"401"," that arrives in the middle of a throttling incident is a different bug entirely, and hammering it looks like credential stuffing to the provider's abuse system — see ",[23,1061,1063],{"href":1062},"\u002Fgetting-started-with-python-apis-for-builders\u002Fparsing-json-responses\u002Fdebugging-401-unauthorized-api-errors\u002F","debugging 401 unauthorized API errors"," for that branch.",[36,1066,1068],{"id":1067},"per-key-or-per-ip-diagnose-before-you-scale","Per-key or per-IP? Diagnose before you scale",[14,1070,1071],{},"This is the fork that decides whether your fix works. A per-key limit follows your credential everywhere; adding machines multiplies the load against a single counter and makes the problem worse. A per-IP limit follows your egress address; if your workers sit behind one NAT gateway or one platform egress pool, ten containers share a single budget and each one thinks it is being reasonable.",[60,1073,69,1078,69,1081,69,1084,69,1087,69,1093,69,1098,69,1102,69,1105,69,1109,69,1112,69,1115,69,1119,69,1122,69,1128,69,1131,69,1135,69,1139,69,1144,69,1148,69,1153,69,1157,69,1162,69,1166,69,1170,69,1174,69,1177,69,1181,69,1185,69,1189],{"viewBox":1074,"role":63,"ariaLabelledBy":1075,"xmlns":67,"style":68},"0 0 720 316",[1076,1077],"d429-scope-t","d429-scope-d",[71,1079,1080],{"id":1076},"Per-key and per-IP counters on the same fleet",[75,1082,1083],{"id":1077},"Three workers sharing one API key and one NAT egress address hit two independent counters at the provider edge; whichever reaches zero first returns the 429.",[79,1085],{"x":81,"y":81,"width":82,"height":1086,"fill":84},"316",[79,1088],{"x":1089,"y":1090,"width":1091,"height":1092,"rx":154,"fill":99,"stroke":100,"style":101},"20","44","150","52",[86,1094,1097],{"x":1095,"y":1096,"fill":111,"style":159},"95","68","worker-1",[86,1099,1101],{"x":1095,"y":1100,"fill":90,"style":164},"85","key K, 8 req\u002Fs",[79,1103],{"x":1089,"y":1104,"width":1091,"height":1092,"rx":154,"fill":99,"stroke":100,"style":101},"120",[86,1106,1108],{"x":1095,"y":1107,"fill":111,"style":159},"144","worker-2",[86,1110,1101],{"x":1095,"y":1111,"fill":90,"style":164},"161",[79,1113],{"x":1089,"y":1114,"width":1091,"height":1092,"rx":154,"fill":99,"stroke":100,"style":101},"196",[86,1116,1118],{"x":1095,"y":1117,"fill":111,"style":159},"220","worker-3",[86,1120,1101],{"x":1095,"y":1121,"fill":90,"style":164},"237",[134,1123],{"x1":1124,"y1":1125,"x2":1126,"y2":1127,"stroke":90,"style":101},"170","70","212","132",[134,1129],{"x1":1124,"y1":1130,"x2":1126,"y2":1130,"stroke":90,"style":101},"146",[134,1132],{"x1":1124,"y1":1133,"x2":1126,"y2":1134,"stroke":90,"style":101},"222","162",[79,1136],{"x":1126,"y":1137,"width":1130,"height":1138,"rx":154,"fill":84,"stroke":139,"style":101},"118","58",[86,1140,1143],{"x":1141,"y":1142,"fill":111,"style":159},"285","142","NAT egress",[86,1145,1147],{"x":1141,"y":1146,"fill":90,"style":164},"160","one shared IP",[134,1149],{"x1":1150,"y1":1151,"x2":1152,"y2":1151,"stroke":90,"style":101},"358","147","404",[79,1154],{"x":1152,"y":89,"width":1155,"height":1156,"rx":98,"fill":99,"stroke":100,"style":101},"296","248",[86,1158,1161],{"x":1159,"y":1160,"fill":90,"style":159},"552","60","provider edge counters",[79,1163],{"x":1164,"y":105,"width":1165,"height":1160,"rx":154,"fill":84,"stroke":144,"style":101},"426","252",[86,1167,1169],{"x":1159,"y":1168,"fill":111,"style":159},"100","per-key budget: 600\u002Fmin",[86,1171,1173],{"x":1159,"y":1172,"fill":90,"style":164},"121","24 req\u002Fs fleet burns it in 25s",[79,1175],{"x":1164,"y":1176,"width":1165,"height":1160,"rx":154,"fill":84,"stroke":148,"style":101},"152",[86,1178,1180],{"x":1159,"y":1179,"fill":111,"style":159},"176","per-IP budget: 1200\u002Fmin",[86,1182,1184],{"x":1159,"y":1183,"fill":90,"style":164},"197","unchanged by adding keys",[86,1186,1188],{"x":1159,"y":1187,"fill":106,"style":159},"244","first counter to hit zero returns 429",[86,1190,1192],{"x":1159,"y":1191,"fill":90,"style":164},"264","the headers tell you which one",[14,1194,1195],{},"Diagnosing this takes ten minutes, not a redesign. Run the same request twice: once from your production egress with a second, unused key, and once from a laptop or a different region with the original key. If the fresh key from the same IP still throttles, the counter is on the address. If the original key throttles from a clean IP, the counter is on the credential. Classify the answer in code so the limiter can react instead of a human.",[207,1197,1199],{"className":209,"code":1198,"language":211,"meta":212,"style":212},"import os\nfrom enum import StrEnum\n\nimport httpx\n\n\nclass LimitScope(StrEnum):\n    KEY = \"key\"\n    ADDRESS = \"address\"\n    UNKNOWN = \"unknown\"\n\n\ndef classify(response: httpx.Response) -> LimitScope:\n    \"\"\"Infer which counter throttled us from the response headers.\"\"\"\n    hinted = response.headers.get(\n        os.getenv(\"RATE_LIMIT_SCOPE_HEADER\", \"x-ratelimit-scope\"), \"\"\n    ).lower()\n    match hinted:\n        case \"api-key\" | \"key\" | \"account\" | \"organization\":\n            return LimitScope.KEY\n        case \"ip\" | \"address\" | \"global\" | \"edge\":\n            return LimitScope.ADDRESS\n        case _:\n            body = response.text[:400].lower()\n            if \"ip address\" in body or \"from this network\" in body:\n                return LimitScope.ADDRESS\n            return LimitScope.KEY if \"x-ratelimit-limit\" in response.headers else LimitScope.UNKNOWN\n",[18,1200,1201,1207,1219,1223,1229,1233,1237,1252,1262,1272,1282,1286,1290,1300,1305,1315,1333,1338,1346,1371,1381,1405,1414,1421,1437,1462,1471],{"__ignoreMap":212},[216,1202,1203,1205],{"class":134,"line":218},[216,1204,222],{"class":221},[216,1206,234],{"class":225},[216,1208,1209,1211,1214,1216],{"class":134,"line":229},[216,1210,248],{"class":221},[216,1212,1213],{"class":225}," enum ",[216,1215,222],{"class":221},[216,1217,1218],{"class":225}," StrEnum\n",[216,1220,1221],{"class":134,"line":237},[216,1222,276],{"emptyLinePlaceholder":275},[216,1224,1225,1227],{"class":134,"line":245},[216,1226,222],{"class":221},[216,1228,284],{"class":225},[216,1230,1231],{"class":134,"line":259},[216,1232,276],{"emptyLinePlaceholder":275},[216,1234,1235],{"class":134,"line":272},[216,1236,276],{"emptyLinePlaceholder":275},[216,1238,1239,1242,1245,1247,1250],{"class":134,"line":279},[216,1240,1241],{"class":221},"class",[216,1243,1244],{"class":360}," LimitScope",[216,1246,422],{"class":225},[216,1248,1249],{"class":360},"StrEnum",[216,1251,513],{"class":225},[216,1253,1254,1257,1259],{"class":134,"line":287},[216,1255,1256],{"class":295},"    KEY",[216,1258,299],{"class":221},[216,1260,1261],{"class":308}," \"key\"\n",[216,1263,1264,1267,1269],{"class":134,"line":292},[216,1265,1266],{"class":295},"    ADDRESS",[216,1268,299],{"class":221},[216,1270,1271],{"class":308}," \"address\"\n",[216,1273,1274,1277,1279],{"class":134,"line":321},[216,1275,1276],{"class":295},"    UNKNOWN",[216,1278,299],{"class":221},[216,1280,1281],{"class":308}," \"unknown\"\n",[216,1283,1284],{"class":134,"line":344},[216,1285,276],{"emptyLinePlaceholder":275},[216,1287,1288],{"class":134,"line":349},[216,1289,276],{"emptyLinePlaceholder":275},[216,1291,1292,1294,1297],{"class":134,"line":354},[216,1293,357],{"class":221},[216,1295,1296],{"class":360}," classify",[216,1298,1299],{"class":225},"(response: httpx.Response) -> LimitScope:\n",[216,1301,1302],{"class":134,"line":391},[216,1303,1304],{"class":308},"    \"\"\"Infer which counter throttled us from the response headers.\"\"\"\n",[216,1306,1307,1310,1312],{"class":134,"line":397},[216,1308,1309],{"class":225},"    hinted ",[216,1311,445],{"class":221},[216,1313,1314],{"class":225}," response.headers.get(\n",[216,1316,1317,1320,1323,1325,1328,1330],{"class":134,"line":413},[216,1318,1319],{"class":225},"        os.getenv(",[216,1321,1322],{"class":308},"\"RATE_LIMIT_SCOPE_HEADER\"",[216,1324,312],{"class":225},[216,1326,1327],{"class":308},"\"x-ratelimit-scope\"",[216,1329,604],{"class":225},[216,1331,1332],{"class":308},"\"\"\n",[216,1334,1335],{"class":134,"line":439},[216,1336,1337],{"class":225},"    ).lower()\n",[216,1339,1340,1343],{"class":134,"line":451},[216,1341,1342],{"class":221},"    match",[216,1344,1345],{"class":225}," hinted:\n",[216,1347,1348,1351,1354,1356,1359,1361,1364,1366,1369],{"class":134,"line":459},[216,1349,1350],{"class":221},"        case",[216,1352,1353],{"class":308}," \"api-key\"",[216,1355,370],{"class":221},[216,1357,1358],{"class":308}," \"key\"",[216,1360,370],{"class":221},[216,1362,1363],{"class":308}," \"account\"",[216,1365,370],{"class":221},[216,1367,1368],{"class":308}," \"organization\"",[216,1370,388],{"class":225},[216,1372,1373,1375,1378],{"class":134,"line":477},[216,1374,706],{"class":221},[216,1376,1377],{"class":225}," LimitScope.",[216,1379,1380],{"class":295},"KEY\n",[216,1382,1383,1385,1388,1390,1393,1395,1398,1400,1403],{"class":134,"line":485},[216,1384,1350],{"class":221},[216,1386,1387],{"class":308}," \"ip\"",[216,1389,370],{"class":221},[216,1391,1392],{"class":308}," \"address\"",[216,1394,370],{"class":221},[216,1396,1397],{"class":308}," \"global\"",[216,1399,370],{"class":221},[216,1401,1402],{"class":308}," \"edge\"",[216,1404,388],{"class":225},[216,1406,1407,1409,1411],{"class":134,"line":496},[216,1408,706],{"class":221},[216,1410,1377],{"class":225},[216,1412,1413],{"class":295},"ADDRESS\n",[216,1415,1416,1418],{"class":134,"line":516},[216,1417,1350],{"class":221},[216,1419,1420],{"class":225}," _:\n",[216,1422,1423,1426,1428,1431,1434],{"class":134,"line":535},[216,1424,1425],{"class":225},"            body ",[216,1427,445],{"class":221},[216,1429,1430],{"class":225}," response.text[:",[216,1432,1433],{"class":295},"400",[216,1435,1436],{"class":225},"].lower()\n",[216,1438,1439,1442,1445,1448,1451,1454,1457,1459],{"class":134,"line":549},[216,1440,1441],{"class":221},"            if",[216,1443,1444],{"class":308}," \"ip address\"",[216,1446,1447],{"class":221}," in",[216,1449,1450],{"class":225}," body ",[216,1452,1453],{"class":221},"or",[216,1455,1456],{"class":308}," \"from this network\"",[216,1458,1447],{"class":221},[216,1460,1461],{"class":225}," body:\n",[216,1463,1464,1467,1469],{"class":134,"line":568},[216,1465,1466],{"class":221},"                return",[216,1468,1377],{"class":225},[216,1470,1413],{"class":295},[216,1472,1473,1475,1477,1480,1483,1486,1488,1491,1494,1496],{"class":134,"line":585},[216,1474,706],{"class":221},[216,1476,1377],{"class":225},[216,1478,1479],{"class":295},"KEY",[216,1481,1482],{"class":221}," if",[216,1484,1485],{"class":308}," \"x-ratelimit-limit\"",[216,1487,1447],{"class":221},[216,1489,1490],{"class":225}," response.headers ",[216,1492,1493],{"class":221},"else",[216,1495,1377],{"class":225},[216,1497,1498],{"class":295},"UNKNOWN\n",[1500,1501,1502,1518],"table",{},[1503,1504,1505],"thead",{},[1506,1507,1508,1512,1515],"tr",{},[1509,1510,1511],"th",{},"Signal",[1509,1513,1514],{},"Per-key limit",[1509,1516,1517],{},"Per-IP limit",[1519,1520,1521,1533,1542,1553],"tbody",{},[1506,1522,1523,1527,1530],{},[1524,1525,1526],"td",{},"Fresh key, same host",[1524,1528,1529],{},"Clears",[1524,1531,1532],{},"Still throttles",[1506,1534,1535,1538,1540],{},[1524,1536,1537],{},"Same key, new host",[1524,1539,1532],{},[1524,1541,1529],{},[1506,1543,1544,1547,1550],{},[1524,1545,1546],{},"Correct fix",[1524,1548,1549],{},"Pace the fleet",[1524,1551,1552],{},"Spread egress",[1506,1554,1555,1558,1561],{},[1524,1556,1557],{},"Cost to fix",[1524,1559,1560],{},"Free",[1524,1562,1563],{},"Proxy or region spend",[14,1565,1566,1567,1571],{},"The commercial consequence sits in that last row. Pacing a fleet against a per-key budget costs you latency and nothing else. Working around a per-IP budget means paying for proxies, extra regions, or a higher provider tier — real recurring spend that belongs in your ",[23,1568,1570],{"href":1569},"\u002Fbuilding-monetizing-api-driven-micro-saas\u002Fdesigning-api-pricing-tiers\u002Fcalculating-cost-per-api-request\u002F","cost per API request"," model before you commit to it. Buying more keys to dodge a documented per-account limit is also the fastest route to a terminated account, so read the terms before you get clever.",[36,1573,1575],{"id":1574},"shape-the-burst-client-side-with-a-token-bucket","Shape the burst client-side with a token bucket",[14,1577,1578,1579,1581],{},"Once you know the budget, stop discovering it by collision. A client-side token bucket converts a spiky workload into a flat one, and flat workloads never see ",[18,1580,20],{}," at all. The numbers matter more than the algorithm. Take a job that fires 82 requests as fast as it can against a 10-requests-per-second budget: the first second sends 40 and 30 bounce, the second sends 38 and 28 bounce, and you have paid for 58 rejected round-trips and a poisoned reputation score to finish two seconds sooner than the shaped version.",[60,1583,69,1588,69,1591,69,1594,69,1597,69,1601,69,1606,69,1608,69,1612,69,1614,69,1617,69,1621,69,1625,69,1630,69,1633,69,1637,69,1640,69,1644,69,1647,69,1650,69,1653,69,1656,69,1659,69,1664,69,1669,69,1673,69,1676,69,1680,69,1684,69,1688,69,1692,69,1696,69,1701,69,1706,69,1709],{"viewBox":1584,"role":63,"ariaLabelledBy":1585,"xmlns":67,"style":68},"0 0 720 328",[1586,1587],"d429-shape-t","d429-shape-d",[71,1589,1590],{"id":1586},"Unshaped burst versus token-bucket shaped output",[75,1592,1593],{"id":1587},"Bar chart of 82 requests sent over eight seconds: the unshaped run spikes to 40 and 38 requests per second above the limit of ten, while the shaped run stays flat at ten.",[79,1595],{"x":81,"y":81,"width":82,"height":1596,"fill":84},"328",[86,1598,1600],{"x":88,"y":1599,"fill":90,"style":91},"26","requests sent per second, 82 total, budget 10\u002Fs",[134,1602],{"x1":1603,"y1":1604,"x2":1605,"y2":1604,"stroke":100,"style":101},"56","272","700",[134,1607],{"x1":1603,"y1":1092,"x2":1603,"y2":1604,"stroke":100,"style":101},[86,1609,104],{"x":95,"y":1610,"fill":90,"style":1611},"66","text-anchor:end;font-size:11;font-family:var(--font-sans);",[86,1613,1089],{"x":95,"y":175,"fill":90,"style":1611},[86,1615,81],{"x":95,"y":1616,"fill":90,"style":1611},"276",[134,1618],{"x1":1603,"y1":1619,"x2":1605,"y2":1619,"stroke":106,"style":1620},"219","stroke-width:2;stroke-dasharray:6 5;",[86,1622,1624],{"x":1623,"y":1126,"fill":106,"style":1611},"694","provider budget",[79,1626],{"x":1610,"y":1627,"width":1628,"height":1629,"fill":106},"62","30","210",[79,1631],{"x":1168,"y":1619,"width":1628,"height":1632,"fill":139},"53",[79,1634],{"x":1107,"y":1635,"width":1628,"height":1636,"fill":106},"73","199",[79,1638],{"x":1639,"y":1619,"width":1628,"height":1632,"fill":139},"178",[79,1641],{"x":1133,"y":1642,"width":1628,"height":1643,"fill":106},"251","21",[79,1645],{"x":1646,"y":1619,"width":1628,"height":1632,"fill":139},"256",[79,1648],{"x":1649,"y":1619,"width":1628,"height":1632,"fill":139},"334",[79,1651],{"x":1652,"y":1619,"width":1628,"height":1632,"fill":139},"412",[79,1654],{"x":1655,"y":1619,"width":1628,"height":1632,"fill":139},"490",[79,1657],{"x":1658,"y":1619,"width":1628,"height":1632,"fill":139},"568",[79,1660],{"x":1661,"y":1662,"width":1628,"height":1663,"fill":139},"646","261","11",[86,1665,1668],{"x":1666,"y":1667,"fill":90,"style":164},"96","290","0s",[86,1670,1672],{"x":1671,"y":1667,"fill":90,"style":164},"174","1s",[86,1674,1675],{"x":1165,"y":1667,"fill":90,"style":164},"2s",[86,1677,1679],{"x":1678,"y":1667,"fill":90,"style":164},"330","3s",[86,1681,1683],{"x":1682,"y":1667,"fill":90,"style":164},"408","4s",[86,1685,1687],{"x":1686,"y":1667,"fill":90,"style":164},"486","5s",[86,1689,1691],{"x":1690,"y":1667,"fill":90,"style":164},"564","6s",[86,1693,1695],{"x":1694,"y":1667,"fill":90,"style":164},"642","7s",[79,1697],{"x":1179,"y":1698,"width":1699,"height":1700,"fill":106},"304","14","12",[86,1702,1705],{"x":1703,"y":1704,"fill":111,"style":91},"198","315","unshaped: 58 rejected",[79,1707],{"x":1708,"y":1698,"width":1699,"height":1700,"fill":139},"392",[86,1710,1712],{"x":1711,"y":1704,"fill":111,"style":91},"414","shaped: 0 rejected",[14,1714,1715,1716,1719],{},"Keep the budget in configuration, not in code, so raising a tier is a deploy of a file rather than a patch. ",[18,1717,1718],{},"tomllib"," is in the standard library on 3.11 and reads a per-provider table cleanly.",[207,1721,1723],{"className":209,"code":1722,"language":211,"meta":212,"style":212},"import asyncio\nimport os\nimport tomllib\nfrom dataclasses import dataclass, field\n\n\n@dataclass\nclass TokenBucket:\n    \"\"\"Async token bucket: `rate` tokens per second, bursting to `capacity`.\"\"\"\n\n    rate: float\n    capacity: float\n    tokens: float = field(init=False)\n    _lock: asyncio.Lock = field(default_factory=asyncio.Lock, init=False)\n\n    def __post_init__(self) -> None:\n        self.tokens = self.capacity\n        self._updated = asyncio.get_event_loop().time()\n\n    async def acquire(self, cost: float = 1.0) -> None:\n        while True:\n            async with self._lock:\n                now = asyncio.get_event_loop().time()\n                self.tokens = min(\n                    self.capacity, self.tokens + (now - self._updated) * self.rate\n                )\n                self._updated = now\n                if self.tokens >= cost:\n                    self.tokens -= cost\n                    return\n                wait = (cost - self.tokens) \u002F self.rate\n            await asyncio.sleep(wait)\n\n\ndef load_bucket(provider: str) -> TokenBucket:\n    with open(os.environ[\"RATE_LIMIT_CONFIG\"], \"rb\") as fh:\n        table = tomllib.load(fh)[\"providers\"][provider]\n    return TokenBucket(rate=float(table[\"rate\"]), capacity=float(table[\"burst\"]))\n",[18,1724,1725,1731,1737,1744,1756,1760,1764,1769,1778,1783,1787,1795,1802,1824,1849,1853,1868,1884,1896,1900,1925,1935,1947,1956,1970,2003,2008,2019,2034,2046,2051,2075,2083,2087,2091,2106,2133,2149],{"__ignoreMap":212},[216,1726,1727,1729],{"class":134,"line":218},[216,1728,222],{"class":221},[216,1730,226],{"class":225},[216,1732,1733,1735],{"class":134,"line":229},[216,1734,222],{"class":221},[216,1736,234],{"class":225},[216,1738,1739,1741],{"class":134,"line":237},[216,1740,222],{"class":221},[216,1742,1743],{"class":225}," tomllib\n",[216,1745,1746,1748,1751,1753],{"class":134,"line":245},[216,1747,248],{"class":221},[216,1749,1750],{"class":225}," dataclasses ",[216,1752,222],{"class":221},[216,1754,1755],{"class":225}," dataclass, field\n",[216,1757,1758],{"class":134,"line":259},[216,1759,276],{"emptyLinePlaceholder":275},[216,1761,1762],{"class":134,"line":272},[216,1763,276],{"emptyLinePlaceholder":275},[216,1765,1766],{"class":134,"line":279},[216,1767,1768],{"class":360},"@dataclass\n",[216,1770,1771,1773,1776],{"class":134,"line":287},[216,1772,1241],{"class":221},[216,1774,1775],{"class":360}," TokenBucket",[216,1777,388],{"class":225},[216,1779,1780],{"class":134,"line":292},[216,1781,1782],{"class":308},"    \"\"\"Async token bucket: `rate` tokens per second, bursting to `capacity`.\"\"\"\n",[216,1784,1785],{"class":134,"line":321},[216,1786,276],{"emptyLinePlaceholder":275},[216,1788,1789,1792],{"class":134,"line":344},[216,1790,1791],{"class":225},"    rate: ",[216,1793,1794],{"class":295},"float\n",[216,1796,1797,1800],{"class":134,"line":349},[216,1798,1799],{"class":225},"    capacity: ",[216,1801,1794],{"class":295},[216,1803,1804,1807,1809,1811,1814,1817,1819,1822],{"class":134,"line":354},[216,1805,1806],{"class":225},"    tokens: ",[216,1808,385],{"class":295},[216,1810,299],{"class":221},[216,1812,1813],{"class":225}," field(",[216,1815,1816],{"class":559},"init",[216,1818,445],{"class":221},[216,1820,1821],{"class":295},"False",[216,1823,436],{"class":225},[216,1825,1826,1829,1831,1833,1836,1838,1841,1843,1845,1847],{"class":134,"line":391},[216,1827,1828],{"class":225},"    _lock: asyncio.Lock ",[216,1830,445],{"class":221},[216,1832,1813],{"class":225},[216,1834,1835],{"class":559},"default_factory",[216,1837,445],{"class":221},[216,1839,1840],{"class":225},"asyncio.Lock, ",[216,1842,1816],{"class":559},[216,1844,445],{"class":221},[216,1846,1821],{"class":295},[216,1848,436],{"class":225},[216,1850,1851],{"class":134,"line":397},[216,1852,276],{"emptyLinePlaceholder":275},[216,1854,1855,1858,1861,1864,1866],{"class":134,"line":413},[216,1856,1857],{"class":221},"    def",[216,1859,1860],{"class":295}," __post_init__",[216,1862,1863],{"class":225},"(self) -> ",[216,1865,827],{"class":295},[216,1867,388],{"class":225},[216,1869,1870,1873,1876,1878,1881],{"class":134,"line":439},[216,1871,1872],{"class":295},"        self",[216,1874,1875],{"class":225},".tokens ",[216,1877,445],{"class":221},[216,1879,1880],{"class":295}," self",[216,1882,1883],{"class":225},".capacity\n",[216,1885,1886,1888,1891,1893],{"class":134,"line":451},[216,1887,1872],{"class":295},[216,1889,1890],{"class":225},"._updated ",[216,1892,445],{"class":221},[216,1894,1895],{"class":225}," asyncio.get_event_loop().time()\n",[216,1897,1898],{"class":134,"line":459},[216,1899,276],{"emptyLinePlaceholder":275},[216,1901,1902,1904,1906,1909,1912,1914,1916,1919,1921,1923],{"class":134,"line":477},[216,1903,925],{"class":221},[216,1905,627],{"class":221},[216,1907,1908],{"class":360}," acquire",[216,1910,1911],{"class":225},"(self, cost: ",[216,1913,385],{"class":295},[216,1915,299],{"class":221},[216,1917,1918],{"class":295}," 1.0",[216,1920,382],{"class":225},[216,1922,827],{"class":295},[216,1924,388],{"class":225},[216,1926,1927,1930,1933],{"class":134,"line":485},[216,1928,1929],{"class":221},"        while",[216,1931,1932],{"class":295}," True",[216,1934,388],{"class":225},[216,1936,1937,1940,1942,1944],{"class":134,"line":496},[216,1938,1939],{"class":221},"            async",[216,1941,928],{"class":221},[216,1943,1880],{"class":295},[216,1945,1946],{"class":225},"._lock:\n",[216,1948,1949,1952,1954],{"class":134,"line":516},[216,1950,1951],{"class":225},"                now ",[216,1953,445],{"class":221},[216,1955,1895],{"class":225},[216,1957,1958,1961,1963,1965,1967],{"class":134,"line":535},[216,1959,1960],{"class":295},"                self",[216,1962,1875],{"class":225},[216,1964,445],{"class":221},[216,1966,419],{"class":295},[216,1968,1969],{"class":225},"(\n",[216,1971,1972,1975,1978,1981,1983,1985,1988,1990,1992,1995,1998,2000],{"class":134,"line":549},[216,1973,1974],{"class":295},"                    self",[216,1976,1977],{"class":225},".capacity, ",[216,1979,1980],{"class":295},"self",[216,1982,1875],{"class":225},[216,1984,745],{"class":221},[216,1986,1987],{"class":225}," (now ",[216,1989,579],{"class":221},[216,1991,1880],{"class":295},[216,1993,1994],{"class":225},"._updated) ",[216,1996,1997],{"class":221},"*",[216,1999,1880],{"class":295},[216,2001,2002],{"class":225},".rate\n",[216,2004,2005],{"class":134,"line":568},[216,2006,2007],{"class":225},"                )\n",[216,2009,2010,2012,2014,2016],{"class":134,"line":585},[216,2011,1960],{"class":295},[216,2013,1890],{"class":225},[216,2015,445],{"class":221},[216,2017,2018],{"class":225}," now\n",[216,2020,2021,2024,2026,2028,2031],{"class":134,"line":611},[216,2022,2023],{"class":221},"                if",[216,2025,1880],{"class":295},[216,2027,1875],{"class":225},[216,2029,2030],{"class":221},">=",[216,2032,2033],{"class":225}," cost:\n",[216,2035,2036,2038,2040,2043],{"class":134,"line":616},[216,2037,1974],{"class":295},[216,2039,1875],{"class":225},[216,2041,2042],{"class":221},"-=",[216,2044,2045],{"class":225}," cost\n",[216,2047,2048],{"class":134,"line":621},[216,2049,2050],{"class":221},"                    return\n",[216,2052,2053,2056,2058,2061,2063,2065,2068,2071,2073],{"class":134,"line":645},[216,2054,2055],{"class":225},"                wait ",[216,2057,445],{"class":221},[216,2059,2060],{"class":225}," (cost ",[216,2062,579],{"class":221},[216,2064,1880],{"class":295},[216,2066,2067],{"class":225},".tokens) ",[216,2069,2070],{"class":221},"\u002F",[216,2072,1880],{"class":295},[216,2074,2002],{"class":225},[216,2076,2077,2080],{"class":134,"line":666},[216,2078,2079],{"class":221},"            await",[216,2081,2082],{"class":225}," asyncio.sleep(wait)\n",[216,2084,2085],{"class":134,"line":680},[216,2086,276],{"emptyLinePlaceholder":275},[216,2088,2089],{"class":134,"line":697},[216,2090,276],{"emptyLinePlaceholder":275},[216,2092,2093,2095,2098,2101,2103],{"class":134,"line":703},[216,2094,357],{"class":221},[216,2096,2097],{"class":360}," load_bucket",[216,2099,2100],{"class":225},"(provider: ",[216,2102,367],{"class":295},[216,2104,2105],{"class":225},") -> TokenBucket:\n",[216,2107,2108,2111,2114,2117,2120,2122,2125,2128,2130],{"class":134,"line":712},[216,2109,2110],{"class":221},"    with",[216,2112,2113],{"class":295}," open",[216,2115,2116],{"class":225},"(os.environ[",[216,2118,2119],{"class":308},"\"RATE_LIMIT_CONFIG\"",[216,2121,947],{"class":225},[216,2123,2124],{"class":308},"\"rb\"",[216,2126,2127],{"class":225},") ",[216,2129,972],{"class":221},[216,2131,2132],{"class":225}," fh:\n",[216,2134,2135,2138,2140,2143,2146],{"class":134,"line":729},[216,2136,2137],{"class":225},"        table ",[216,2139,445],{"class":221},[216,2141,2142],{"class":225}," tomllib.load(fh)[",[216,2144,2145],{"class":308},"\"providers\"",[216,2147,2148],{"class":225},"][provider]\n",[216,2150,2151,2153,2156,2159,2161,2163,2166,2169,2172,2175,2177,2179,2181,2184],{"class":134,"line":736},[216,2152,588],{"class":221},[216,2154,2155],{"class":225}," TokenBucket(",[216,2157,2158],{"class":559},"rate",[216,2160,445],{"class":221},[216,2162,385],{"class":295},[216,2164,2165],{"class":225},"(table[",[216,2167,2168],{"class":308},"\"rate\"",[216,2170,2171],{"class":225},"]), ",[216,2173,2174],{"class":559},"capacity",[216,2176,445],{"class":221},[216,2178,385],{"class":295},[216,2180,2165],{"class":225},[216,2182,2183],{"class":308},"\"burst\"",[216,2185,2186],{"class":225},"]))\n",[14,2188,2189,2190,2192,2193,2197],{},"Set ",[18,2191,2158],{}," to about 80 percent of the published budget. The remaining headroom absorbs clock skew between your host and the provider's window boundary, plus whatever other process on your account also holds that key. If you run a job queue, put the bucket inside the worker rather than the producer — ",[23,2194,2196],{"href":2195},"\u002Fscaling-and-operating-production-python-apis\u002Frunning-background-jobs-with-celery\u002F","running background jobs with Celery"," shows where that boundary sits.",[36,2199,2201],{"id":2200},"coordinate-the-fleet-through-one-shared-counter","Coordinate the fleet through one shared counter",[14,2203,2204,2205,2209],{},"An in-process bucket is correct for one process and wrong for three. Three replicas each pacing at the full budget send triple the budget. Move the counter to Redis, evaluate it atomically in Lua so two workers cannot both read the same token, and have the script return the wait time rather than a boolean — a boolean forces a polling loop, a wait time lets the caller sleep exactly once. The same Redis instance you already run for ",[23,2206,2208],{"href":2207},"\u002Fscaling-and-operating-production-python-apis\u002Fcaching-python-api-responses-with-redis\u002F","caching Python API responses"," handles this without extra infrastructure.",[60,2211,69,2216,69,2219,69,2222,69,2224,69,2228,69,2230,69,2233,69,2236,69,2239,69,2243,69,2246,69,2250,69,2254,69,2256,69,2258,69,2260,69,2262,69,2266,69,2268,69,2272,69,2275,69,2280,69,2282,69,2285,69,2288,69,2292,69,2295],{"viewBox":2212,"role":63,"ariaLabelledBy":2213,"xmlns":67,"style":68},"0 0 720 300",[2214,2215],"d429-fleet-t","d429-fleet-d",[71,2217,2218],{"id":2214},"Two workers coordinating through a shared Redis bucket",[75,2220,2221],{"id":2215},"Sequence diagram: worker one takes a token from Redis and calls the provider, while worker two is told to wait 240 milliseconds instead of sending a request that would return 429.",[79,2223],{"x":81,"y":81,"width":82,"height":152,"fill":84},[79,2225],{"x":88,"y":1089,"width":2226,"height":2227,"rx":154,"fill":99,"stroke":100,"style":101},"140","38",[86,2229,1097],{"x":158,"y":1090,"fill":111,"style":159},[79,2231],{"x":2232,"y":1089,"width":2226,"height":2227,"rx":154,"fill":99,"stroke":100,"style":101},"204",[86,2234,1108],{"x":2235,"y":1090,"fill":111,"style":159},"274",[79,2237],{"x":2238,"y":1089,"width":2226,"height":2227,"rx":154,"fill":84,"stroke":139,"style":101},"384",[86,2240,2242],{"x":2241,"y":1090,"fill":111,"style":159},"454","redis bucket",[79,2244],{"x":2245,"y":1089,"width":2226,"height":2227,"rx":154,"fill":84,"stroke":144,"style":101},"556",[86,2247,2249],{"x":2248,"y":1090,"fill":111,"style":159},"626","provider",[134,2251],{"x1":158,"y1":1138,"x2":158,"y2":2252,"stroke":100,"style":2253},"278","stroke-width:2;stroke-dasharray:4 5;",[134,2255],{"x1":2235,"y1":1138,"x2":2235,"y2":2252,"stroke":100,"style":2253},[134,2257],{"x1":2241,"y1":1138,"x2":2241,"y2":2252,"stroke":100,"style":2253},[134,2259],{"x1":2248,"y1":1138,"x2":2248,"y2":2252,"stroke":100,"style":2253},[134,2261],{"x1":158,"y1":1666,"x2":2241,"y2":1666,"stroke":139,"style":101},[86,2263,2265],{"x":2235,"y":2264,"fill":111,"style":159},"88","take 1 token",[134,2267],{"x1":2241,"y1":168,"x2":158,"y2":168,"stroke":139,"style":101},[86,2269,2271],{"x":2235,"y":2270,"fill":90,"style":164},"126","wait 0.0s, 3 tokens left",[134,2273],{"x1":158,"y1":2274,"x2":2248,"y2":2274,"stroke":144,"style":101},"172",[86,2276,2279],{"x":2277,"y":2278,"fill":111,"style":159},"360","164","GET \u002Fv1\u002Fitems → 200 OK",[134,2281],{"x1":2235,"y1":1126,"x2":2241,"y2":1126,"stroke":139,"style":101},[86,2283,2265],{"x":2284,"y":2232,"fill":111,"style":159},"364",[134,2286],{"x1":2241,"y1":2287,"x2":2235,"y2":2287,"stroke":106,"style":101},"250",[86,2289,2291],{"x":2284,"y":2290,"fill":106,"style":164},"242","wait 0.24s, bucket empty",[86,2293,2294],{"x":158,"y":1604,"fill":90,"style":164},"sends",[86,2296,2297],{"x":2235,"y":1604,"fill":90,"style":164},"sleeps, no 429",[207,2299,2301],{"className":209,"code":2300,"language":211,"meta":212,"style":212},"import asyncio\nimport os\n\nimport redis.asyncio as redis\n\nBUCKET_LUA = \"\"\"\nlocal tokens = tonumber(redis.call('HGET', KEYS[1], 'tokens'))\nlocal ts = tonumber(redis.call('HGET', KEYS[1], 'ts'))\nlocal rate, capacity, now, cost = tonumber(ARGV[1]), tonumber(ARGV[2]), tonumber(ARGV[3]), tonumber(ARGV[4])\nif tokens == nil then tokens = capacity; ts = now end\ntokens = math.min(capacity, tokens + math.max(0, now - ts) * rate)\nlocal wait = 0\nif tokens >= cost then tokens = tokens - cost else wait = (cost - tokens) \u002F rate end\nredis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)\nredis.call('EXPIRE', KEYS[1], 300)\nreturn tostring(wait)\n\"\"\"\n\n\nclass FleetLimiter:\n    def __init__(self, client: redis.Redis, key: str, rate: float, burst: float):\n        self._script = client.register_script(BUCKET_LUA)\n        self._key, self._rate, self._burst = key, rate, burst\n\n    async def acquire(self, cost: float = 1.0) -> None:\n        loop = asyncio.get_running_loop()\n        while True:\n            raw = await self._script(\n                keys=[self._key], args=[self._rate, self._burst, loop.time(), cost]\n            )\n            wait = float(raw)\n            if wait \u003C= 0:\n                return\n            await asyncio.sleep(wait)\n\n\nasync def build_limiter() -> FleetLimiter:\n    client = redis.from_url(os.environ[\"REDIS_URL\"], decode_responses=True)\n    return FleetLimiter(\n        client,\n        key=os.getenv(\"RATE_LIMIT_KEY\", \"ratelimit:provider\"),\n        rate=float(os.getenv(\"RATE_LIMIT_RATE\", \"8\")),\n        burst=float(os.getenv(\"RATE_LIMIT_BURST\", \"16\")),\n    )\n",[18,2302,2303,2309,2315,2319,2331,2335,2345,2350,2355,2360,2365,2370,2375,2380,2385,2390,2395,2400,2404,2408,2417,2441,2457,2479,2483,2505,2515,2523,2537,2568,2573,2585,2600,2605,2611,2615,2619,2631,2656,2663,2668,2689,2711,2732],{"__ignoreMap":212},[216,2304,2305,2307],{"class":134,"line":218},[216,2306,222],{"class":221},[216,2308,226],{"class":225},[216,2310,2311,2313],{"class":134,"line":229},[216,2312,222],{"class":221},[216,2314,234],{"class":225},[216,2316,2317],{"class":134,"line":237},[216,2318,276],{"emptyLinePlaceholder":275},[216,2320,2321,2323,2326,2328],{"class":134,"line":245},[216,2322,222],{"class":221},[216,2324,2325],{"class":225}," redis.asyncio ",[216,2327,972],{"class":221},[216,2329,2330],{"class":225}," redis\n",[216,2332,2333],{"class":134,"line":259},[216,2334,276],{"emptyLinePlaceholder":275},[216,2336,2337,2340,2342],{"class":134,"line":272},[216,2338,2339],{"class":295},"BUCKET_LUA",[216,2341,299],{"class":221},[216,2343,2344],{"class":308}," \"\"\"\n",[216,2346,2347],{"class":134,"line":279},[216,2348,2349],{"class":308},"local tokens = tonumber(redis.call('HGET', KEYS[1], 'tokens'))\n",[216,2351,2352],{"class":134,"line":287},[216,2353,2354],{"class":308},"local ts = tonumber(redis.call('HGET', KEYS[1], 'ts'))\n",[216,2356,2357],{"class":134,"line":292},[216,2358,2359],{"class":308},"local rate, capacity, now, cost = tonumber(ARGV[1]), tonumber(ARGV[2]), tonumber(ARGV[3]), tonumber(ARGV[4])\n",[216,2361,2362],{"class":134,"line":321},[216,2363,2364],{"class":308},"if tokens == nil then tokens = capacity; ts = now end\n",[216,2366,2367],{"class":134,"line":344},[216,2368,2369],{"class":308},"tokens = math.min(capacity, tokens + math.max(0, now - ts) * rate)\n",[216,2371,2372],{"class":134,"line":349},[216,2373,2374],{"class":308},"local wait = 0\n",[216,2376,2377],{"class":134,"line":354},[216,2378,2379],{"class":308},"if tokens >= cost then tokens = tokens - cost else wait = (cost - tokens) \u002F rate end\n",[216,2381,2382],{"class":134,"line":391},[216,2383,2384],{"class":308},"redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)\n",[216,2386,2387],{"class":134,"line":397},[216,2388,2389],{"class":308},"redis.call('EXPIRE', KEYS[1], 300)\n",[216,2391,2392],{"class":134,"line":413},[216,2393,2394],{"class":308},"return tostring(wait)\n",[216,2396,2397],{"class":134,"line":439},[216,2398,2399],{"class":308},"\"\"\"\n",[216,2401,2402],{"class":134,"line":451},[216,2403,276],{"emptyLinePlaceholder":275},[216,2405,2406],{"class":134,"line":459},[216,2407,276],{"emptyLinePlaceholder":275},[216,2409,2410,2412,2415],{"class":134,"line":477},[216,2411,1241],{"class":221},[216,2413,2414],{"class":360}," FleetLimiter",[216,2416,388],{"class":225},[216,2418,2419,2421,2424,2427,2429,2432,2434,2437,2439],{"class":134,"line":485},[216,2420,1857],{"class":221},[216,2422,2423],{"class":295}," __init__",[216,2425,2426],{"class":225},"(self, client: redis.Redis, key: ",[216,2428,367],{"class":295},[216,2430,2431],{"class":225},", rate: ",[216,2433,385],{"class":295},[216,2435,2436],{"class":225},", burst: ",[216,2438,385],{"class":295},[216,2440,513],{"class":225},[216,2442,2443,2445,2448,2450,2453,2455],{"class":134,"line":496},[216,2444,1872],{"class":295},[216,2446,2447],{"class":225},"._script ",[216,2449,445],{"class":221},[216,2451,2452],{"class":225}," client.register_script(",[216,2454,2339],{"class":295},[216,2456,436],{"class":225},[216,2458,2459,2461,2464,2466,2469,2471,2474,2476],{"class":134,"line":516},[216,2460,1872],{"class":295},[216,2462,2463],{"class":225},"._key, ",[216,2465,1980],{"class":295},[216,2467,2468],{"class":225},"._rate, ",[216,2470,1980],{"class":295},[216,2472,2473],{"class":225},"._burst ",[216,2475,445],{"class":221},[216,2477,2478],{"class":225}," key, rate, burst\n",[216,2480,2481],{"class":134,"line":535},[216,2482,276],{"emptyLinePlaceholder":275},[216,2484,2485,2487,2489,2491,2493,2495,2497,2499,2501,2503],{"class":134,"line":549},[216,2486,925],{"class":221},[216,2488,627],{"class":221},[216,2490,1908],{"class":360},[216,2492,1911],{"class":225},[216,2494,385],{"class":295},[216,2496,299],{"class":221},[216,2498,1918],{"class":295},[216,2500,382],{"class":225},[216,2502,827],{"class":295},[216,2504,388],{"class":225},[216,2506,2507,2510,2512],{"class":134,"line":568},[216,2508,2509],{"class":225},"        loop ",[216,2511,445],{"class":221},[216,2513,2514],{"class":225}," asyncio.get_running_loop()\n",[216,2516,2517,2519,2521],{"class":134,"line":585},[216,2518,1929],{"class":221},[216,2520,1932],{"class":295},[216,2522,388],{"class":225},[216,2524,2525,2528,2530,2532,2534],{"class":134,"line":611},[216,2526,2527],{"class":225},"            raw ",[216,2529,445],{"class":221},[216,2531,674],{"class":221},[216,2533,1880],{"class":295},[216,2535,2536],{"class":225},"._script(\n",[216,2538,2539,2542,2544,2547,2549,2552,2555,2557,2559,2561,2563,2565],{"class":134,"line":616},[216,2540,2541],{"class":559},"                keys",[216,2543,445],{"class":221},[216,2545,2546],{"class":225},"[",[216,2548,1980],{"class":295},[216,2550,2551],{"class":225},"._key], ",[216,2553,2554],{"class":559},"args",[216,2556,445],{"class":221},[216,2558,2546],{"class":225},[216,2560,1980],{"class":295},[216,2562,2468],{"class":225},[216,2564,1980],{"class":295},[216,2566,2567],{"class":225},"._burst, loop.time(), cost]\n",[216,2569,2570],{"class":134,"line":621},[216,2571,2572],{"class":225},"            )\n",[216,2574,2575,2578,2580,2582],{"class":134,"line":645},[216,2576,2577],{"class":225},"            wait ",[216,2579,445],{"class":221},[216,2581,302],{"class":295},[216,2583,2584],{"class":225},"(raw)\n",[216,2586,2587,2589,2592,2595,2598],{"class":134,"line":666},[216,2588,1441],{"class":221},[216,2590,2591],{"class":225}," wait ",[216,2593,2594],{"class":221},"\u003C=",[216,2596,2597],{"class":295}," 0",[216,2599,388],{"class":225},[216,2601,2602],{"class":134,"line":680},[216,2603,2604],{"class":221},"                return\n",[216,2606,2607,2609],{"class":134,"line":697},[216,2608,2079],{"class":221},[216,2610,2082],{"class":225},[216,2612,2613],{"class":134,"line":703},[216,2614,276],{"emptyLinePlaceholder":275},[216,2616,2617],{"class":134,"line":712},[216,2618,276],{"emptyLinePlaceholder":275},[216,2620,2621,2623,2625,2628],{"class":134,"line":729},[216,2622,624],{"class":221},[216,2624,627],{"class":221},[216,2626,2627],{"class":360}," build_limiter",[216,2629,2630],{"class":225},"() -> FleetLimiter:\n",[216,2632,2633,2636,2638,2641,2644,2646,2649,2651,2654],{"class":134,"line":736},[216,2634,2635],{"class":225},"    client ",[216,2637,445],{"class":221},[216,2639,2640],{"class":225}," redis.from_url(os.environ[",[216,2642,2643],{"class":308},"\"REDIS_URL\"",[216,2645,947],{"class":225},[216,2647,2648],{"class":559},"decode_responses",[216,2650,445],{"class":221},[216,2652,2653],{"class":295},"True",[216,2655,436],{"class":225},[216,2657,2658,2660],{"class":134,"line":767},[216,2659,588],{"class":221},[216,2661,2662],{"class":225}," FleetLimiter(\n",[216,2664,2665],{"class":134,"line":804},[216,2666,2667],{"class":225},"        client,\n",[216,2669,2670,2673,2675,2678,2681,2683,2686],{"class":134,"line":809},[216,2671,2672],{"class":559},"        key",[216,2674,445],{"class":221},[216,2676,2677],{"class":225},"os.getenv(",[216,2679,2680],{"class":308},"\"RATE_LIMIT_KEY\"",[216,2682,312],{"class":225},[216,2684,2685],{"class":308},"\"ratelimit:provider\"",[216,2687,2688],{"class":225},"),\n",[216,2690,2691,2694,2696,2698,2700,2703,2705,2708],{"class":134,"line":814},[216,2692,2693],{"class":559},"        rate",[216,2695,445],{"class":221},[216,2697,385],{"class":295},[216,2699,305],{"class":225},[216,2701,2702],{"class":308},"\"RATE_LIMIT_RATE\"",[216,2704,312],{"class":225},[216,2706,2707],{"class":308},"\"8\"",[216,2709,2710],{"class":225},")),\n",[216,2712,2713,2716,2718,2720,2722,2725,2727,2730],{"class":134,"line":832},[216,2714,2715],{"class":559},"        burst",[216,2717,445],{"class":221},[216,2719,385],{"class":295},[216,2721,305],{"class":225},[216,2723,2724],{"class":308},"\"RATE_LIMIT_BURST\"",[216,2726,312],{"class":225},[216,2728,2729],{"class":308},"\"16\"",[216,2731,2710],{"class":225},[216,2733,2734],{"class":134,"line":872},[216,2735,2736],{"class":225},"    )\n",[14,2738,2739,2740,2743,2744,2748],{},"Use a wall-clock source shared across hosts in production rather than each loop's monotonic clock; ",[18,2741,2742],{},"time.time()"," with NTP is accurate enough for a limiter measured in hundreds of milliseconds. Emit the returned wait value as a metric. Rising wait times are the leading indicator that you need a higher tier, and ",[23,2745,2747],{"href":2746},"\u002Fscaling-and-operating-production-python-apis\u002Fmonitoring-and-logging-python-apis\u002Fstructured-logging-with-structlog\u002F","structured logging with structlog"," makes that a one-line dashboard query.",[36,2750,2752],{"id":2751},"when-to-choose-each-fix-and-when-to-skip-it","When to choose each fix, and when to skip it",[14,2754,2755,2756,2758],{},"Retry-only is right when ",[18,2757,20],{}," is rare — under one percent of calls — and your workload is a handful of interactive requests. Adding a limiter there costs latency on every single call to save you a retry you almost never perform. An in-process token bucket wins the moment you have one batch job that fans out, and it stays right until you run more than one replica. The Redis-backed fleet limiter earns its operational cost at three or more workers, or any autoscaling group where the replica count is not something you control.",[14,2760,2761],{},"Avoid the distributed limiter when throughput is genuinely small: a shared counter adds one to three milliseconds to every outbound call and one more service that can take your pipeline down. Avoid buying proxy egress until you have proven the counter is on the address, because that spend recurs monthly.",[36,2763,2765],{"id":2764},"migration-path-from-blind-retries-to-a-shaped-fleet","Migration path from blind retries to a shaped fleet",[14,2767,2768,2769,2771,2772,2774,2775,2778,2779,2782,2783,2787],{},"Do this in four steps, each shippable on its own. First, log the full header set on every ",[18,2770,20],{}," for a day and classify the scope — this is the diagnosis and it costs you one deploy. Second, replace the fixed sleep with ",[18,2773,33],{}," parsing and jitter; this alone typically halves the error rate because you stop retrying into a closed window. Third, drop the in-process bucket in front of the client and set the rate from the observed ",[18,2776,2777],{},"x-ratelimit-limit"," at 80 percent. Fourth, when you scale past one replica, swap the bucket implementation for the Redis-backed one — keep the ",[18,2780,2781],{},"acquire()"," signature identical and the call sites never change. If a key rotation is on your roadmap, sequence it after step three, since ",[23,2784,2786],{"href":2785},"\u002Fgetting-started-with-python-apis-for-builders\u002Fhandling-api-authentication-in-python\u002Frotating-api-keys-without-downtime\u002F","rotating API keys without downtime"," briefly doubles your key count and confuses a per-key counter.",[36,2789,2791],{"id":2790},"builder-verdict","Builder verdict",[14,2793,2794],{},"Ship the Redis-backed token bucket. Not on day one, but as the target you migrate toward, because it is the only option whose cost stays flat as you scale. Retry loops are free to write and expensive to run: every rejected call still costs you a TCP handshake, a TLS negotiation, and a slot in your connection pool, and at a hundred thousand calls a day a five percent rejection rate is five thousand wasted round-trips that also degrade how the provider scores your traffic. The in-process bucket is the right stopgap for a single-container side project and takes twenty minutes. The distributed version takes an afternoon and one Redis key you probably already pay for, and it removes an entire class of 3am pages. The one thing not worth doing is minting extra API keys to route around a limit: it works for a month, then the provider notices, and rebuilding under a suspension notice costs more than every other option here combined.",[36,2796,2798],{"id":2797},"faq","FAQ",[14,2800,2801,2805],{},[2802,2803,2804],"strong",{},"How much does a 429 storm actually cost me at 1M requests a month?","\nRejected calls are not free even though they return no data. At a five percent rejection rate you pay for 50,000 extra TLS handshakes, roughly 40 to 60 seconds of aggregate worker CPU, and the retry traffic itself. On a small container that is a couple of dollars a month in compute, but the real cost is the throughput ceiling: workers blocked in backoff are workers not doing billable work, so you provision more of them than you need.",[14,2807,2808,2811],{},[2802,2809,2810],{},"Is it cheaper to pace my requests or to buy a higher provider tier?","\nPace first, always. Shaping costs you engineering hours once and zero recurring spend, and it also tells you your true required throughput. Once a shaped client still saturates its budget for hours a day, you have hard evidence for the upgrade, and you can price the tier against the revenue that traffic produces instead of guessing.",[14,2813,2814,2817],{},[2802,2815,2816],{},"Will rotating my API key reset the rate limit counter?","\nNo, for any provider worth integrating. Limits are attached to the account or project, not the individual credential, so a fresh key inherits the same budget. Rotation that appears to reset the counter usually means the counter is per-key on a provider that also enforces a per-account ceiling you have not hit yet, and relying on that is a good way to lose the account.",[14,2819,2820,2823],{},[2802,2821,2822],{},"How risky is migrating from a per-process limiter to a shared Redis one?","\nLow, if you keep the interface identical. The failure mode to plan for is Redis being unreachable: decide upfront whether the limiter fails open (send anyway, risk 429) or fails closed (block, risk a stalled queue). For paid third-party APIs, fail open with a conservative local fallback bucket — a stalled pipeline usually costs more than a handful of rejected calls.",[14,2825,2826,2829,2830,2834,2835,2838],{},[2802,2827,2828],{},"Does this apply to the API I sell, or only the ones I consume?","\nBoth, from opposite sides. Everything here describes being a good client; the mirror image is enforcing limits on your own customers, which is where tier design and ",[23,2831,2833],{"href":2832},"\u002Fbuilding-monetizing-api-driven-micro-saas\u002Fdesigning-api-pricing-tiers\u002Fpreventing-free-tier-abuse\u002F","preventing free-tier abuse"," come in. Send the same headers you wish providers sent you — a documented ",[18,2836,2837],{},"retry-after"," and an honest scope hint cut your support load noticeably.",[36,2840,2842],{"id":2841},"related","Related",[2844,2845,2846,2852,2859,2865,2872],"ul",{},[2847,2848,2849,2851],"li",{},[23,2850,26],{"href":25}," — the parent guide covering sessions, timeouts, and connection reuse.",[2847,2853,2854,2858],{},[23,2855,2857],{"href":2856},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fbest-practices-for-api-rate-limiting\u002F","Best Practices for API Rate Limiting"," — the broader strategy this debugging playbook implements.",[2847,2860,2861,2864],{},[23,2862,2863],{"href":1043},"Retrying Failed HTTP Requests with tenacity"," — turn the hand-rolled retry loop above into a declarative policy.",[2847,2866,2867,2871],{},[23,2868,2870],{"href":2869},"\u002Fscaling-and-operating-production-python-apis\u002Fcaching-python-api-responses-with-redis\u002Fredis-vs-in-memory-caching-for-fastapi\u002F","Redis vs In-Memory Caching for FastAPI"," — the same shared-state trade-off applied to cached responses.",[2847,2873,2874,2878],{},[23,2875,2877],{"href":2876},"\u002Fautomating-side-hustle-operations-with-apis\u002Fautomating-ai-workflows-with-python-apis\u002Fcontrolling-llm-api-costs-in-production\u002F","Controlling LLM API Costs in Production"," — token-per-minute limits are the same problem with a much bigger bill attached.",[2880,2881,2882],"style",{},"html pre.shiki code .szBVR, html code.shiki .szBVR{--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .sVt8B, html code.shiki .sVt8B{--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .s4XuR, html code.shiki .s4XuR{--shiki-default:#E36209;--shiki-dark:#FFAB70}html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":212,"searchDepth":229,"depth":229,"links":2884},[2885,2886,2887,2888,2889,2890,2891,2892,2893,2894],{"id":38,"depth":229,"text":39},{"id":193,"depth":229,"text":194},{"id":1067,"depth":229,"text":1068},{"id":1574,"depth":229,"text":1575},{"id":2200,"depth":229,"text":2201},{"id":2751,"depth":229,"text":2752},{"id":2764,"depth":229,"text":2765},{"id":2790,"depth":229,"text":2791},{"id":2797,"depth":229,"text":2798},{"id":2841,"depth":229,"text":2842},"Fix 429 Too Many Requests in Python: read rate-limit headers, honour Retry-After, separate per-key from per-IP limits, and shape bursts across a worker fleet.","md",{"pageTitle":2898,"type":2899,"datePublished":2900,"dateModified":2900},"Debugging 429 Too Many Requests Errors: Python Playbook","article","2026-07-23","\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fdebugging-429-too-many-requests-errors",{"title":5,"description":2895},"getting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fdebugging-429-too-many-requests-errors\u002Findex","QbhnhEufNBmYS0-K6Tw8OURsSAs066ZL9yQTJQK4fms",{"@context":2906,"@type":2907,"mainEntity":2908},"https:\u002F\u002Fschema.org","FAQPage",[2909,2914,2917,2920,2923],{"@type":2910,"name":2804,"acceptedAnswer":2911},"Question",{"@type":2912,"text":2913},"Answer","Rejected calls are not free even though they return no data. At a five percent rejection rate you pay for 50,000 extra TLS handshakes, roughly 40 to 60 seconds of aggregate worker CPU, and the retry traffic itself. On a small container that is a couple of dollars a month in compute, but the real cost is the throughput ceiling: workers blocked in backoff are workers not doing billable work, so you provision more of them than you need.",{"@type":2910,"name":2810,"acceptedAnswer":2915},{"@type":2912,"text":2916},"Pace first, always. Shaping costs you engineering hours once and zero recurring spend, and it also tells you your true required throughput. Once a shaped client still saturates its budget for hours a day, you have hard evidence for the upgrade, and you can price the tier against the revenue that traffic produces instead of guessing.",{"@type":2910,"name":2816,"acceptedAnswer":2918},{"@type":2912,"text":2919},"No, for any provider worth integrating. Limits are attached to the account or project, not the individual credential, so a fresh key inherits the same budget. Rotation that appears to reset the counter usually means the counter is per-key on a provider that also enforces a per-account ceiling you have not hit yet, and relying on that is a good way to lose the account.",{"@type":2910,"name":2822,"acceptedAnswer":2921},{"@type":2912,"text":2922},"Low, if you keep the interface identical. The failure mode to plan for is Redis being unreachable: decide upfront whether the limiter fails open (send anyway, risk 429) or fails closed (block, risk a stalled queue). For paid third-party APIs, fail open with a conservative local fallback bucket — a stalled pipeline usually costs more than a handful of rejected calls.",{"@type":2910,"name":2828,"acceptedAnswer":2924},{"@type":2912,"text":2925},"Both, from opposite sides. Everything here describes being a good client; the mirror image is enforcing limits on your own customers, which is where tier design and preventing free-tier abuse come in. Send the same headers you wish providers sent you — a documented retry-after and an honest scope hint cut your support load noticeably.",1784887028359]