[{"data":1,"prerenderedAt":2305},["ShallowReactive",2],{"page-\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fhttpx-vs-requests-for-async\u002F":3,"faq-schema-\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fhttpx-vs-requests-for-async\u002F":2284},{"id":4,"title":5,"body":6,"description":2273,"extension":2274,"meta":2275,"navigation":256,"path":2280,"seo":2281,"stem":2282,"__hash__":2283},"content\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fhttpx-vs-requests-for-async\u002Findex.md","httpx vs requests: Choosing an HTTP Client for Async Python",{"type":7,"value":8,"toc":2261},"minimark",[9,13,31,41,44,49,54,74,178,180,184,200,377,387,469,479,481,485,497,939,954,956,960,981,1000,1255,1277,1375,1377,1381,1395,1398,1796,1810,1921,1923,1927,1930,1946,1970,1987,2003,2022,2024,2028,2031,2112,2130,2138,2140,2144,2162,2166,2172,2178,2187,2196,2208,2212,2217,2238,2243,2257],[10,11,5],"h1",{"id":12},"httpx-vs-requests-choosing-an-http-client-for-async-python",[14,15,16,17,21,22,24,25,30],"p",{},"The ",[18,19,20],"code",{},"requests"," library is the comfortable default for calling APIs from Python, and for synchronous scripts it is still excellent. But the moment your code runs inside an async framework, ",[18,23,20],{}," becomes a liability — it blocks the event loop and silently erases the concurrency you adopted async to get. This page resolves one decision: which HTTP client belongs in a commercial Python API that calls other APIs. Part of the ",[26,27,29],"a",{"href":28},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002F","Making HTTP Requests with the requests Library"," guide.",[14,32,33,34,37,38,40],{},"The short version: use ",[18,35,36],{},"httpx.AsyncClient"," for anything inside an event loop, create exactly one client for the lifetime of the process, set every timeout phase explicitly against your own latency budget, and size the connection pool to what your upstream will actually tolerate. Keep ",[18,39,20],{}," for scripts and cron jobs where blocking is free.",[42,43],"hr",{},[45,46,48],"h2",{"id":47},"the-core-difference-one-execution-model-versus-two","The core difference: one execution model versus two",[14,50,51,53],{},[18,52,20],{}," has exactly one execution model: synchronous and blocking. Every call sits on the calling thread until the server responds. That is perfectly fine in a script, a one-off data pull, or a thread-per-request server, and it has been the dependable choice for a decade precisely because it does one thing predictably.",[14,55,56,59,60,63,64,66,67,69,70,73],{},[18,57,58],{},"httpx"," offers the same familiar surface in two flavours. ",[18,61,62],{},"httpx.Client"," is a synchronous client whose method names mirror ",[18,65,20],{},", so most existing code ports with a search-and-replace. ",[18,68,36],{}," is the async version, and it is the reason the library exists: ",[18,71,72],{},"await"," hands control back to the event loop while the socket waits, so one loop can hold hundreds of requests in flight at once. That single capability is the whole argument.",[75,76,84,85,84,89,84,93,84,100,84,108,84,118,84,125,84,130,84,135,84,140,84,143,84,147,84,151,84,155,84,159,84,162,84,165,84,167,84,170,84,172,84,175],"svg",{"viewBox":77,"role":78,"ariaLabelledBy":79,"xmlns":82,"style":83},"0 0 720 300","img",[80,81],"hvr-loop-t","hvr-loop-d","http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","width:100%;height:auto;margin:1.5rem 0;font-family:var(--font-sans);","\n  ",[86,87,88],"title",{"id":80},"Blocking requests versus awaited httpx in one event loop",[90,91,92],"desc",{"id":81},"A blocking requests call holds the event loop until it returns so queued requests wait, while httpx AsyncClient yields control and three requests progress concurrently.",[94,95],"rect",{"x":96,"y":96,"width":97,"height":98,"fill":99},"0","720","300","var(--c-surface)",[101,102,107],"text",{"x":103,"y":104,"fill":105,"style":106},"180","30","var(--c-text)","text-anchor:middle;font-size:14;font-family:var(--font-sans);","requests inside async def",[94,109],{"x":110,"y":111,"width":112,"height":113,"rx":114,"fill":115,"stroke":116,"style":117},"40","46","280","196","12","var(--c-surface-alt)","var(--c-coral)","stroke-width:2;",[94,119],{"x":120,"y":121,"width":122,"height":110,"rx":123,"fill":99,"stroke":116,"style":124},"64","70","232","7","stroke-width:1.5;",[101,126,129],{"x":103,"y":127,"fill":105,"style":128},"95","text-anchor:middle;font-size:12;font-family:var(--font-sans);","Request A holds the loop",[94,131],{"x":120,"y":132,"width":122,"height":110,"rx":123,"fill":99,"stroke":133,"style":134},"120","var(--c-border)","stroke-width:1.5;stroke-dasharray:5 4;",[101,136,139],{"x":103,"y":137,"fill":138,"style":128},"145","var(--c-text-muted)","Request B queued",[94,141],{"x":120,"y":142,"width":122,"height":110,"rx":123,"fill":99,"stroke":133,"style":134},"170",[101,144,146],{"x":103,"y":145,"fill":138,"style":128},"195","Request C queued",[101,148,150],{"x":103,"y":149,"fill":138,"style":128},"268","1 call in flight per worker",[101,152,154],{"x":153,"y":104,"fill":105,"style":106},"540","httpx.AsyncClient awaited",[94,156],{"x":157,"y":111,"width":112,"height":113,"rx":114,"fill":115,"stroke":158,"style":117},"400","var(--c-teal)",[94,160],{"x":161,"y":121,"width":122,"height":110,"rx":123,"fill":99,"stroke":158,"style":124},"424",[101,163,164],{"x":153,"y":127,"fill":105,"style":128},"Request A awaits socket",[94,166],{"x":161,"y":132,"width":122,"height":110,"rx":123,"fill":99,"stroke":158,"style":124},[101,168,169],{"x":153,"y":137,"fill":105,"style":128},"Request B runs too",[94,171],{"x":161,"y":142,"width":122,"height":110,"rx":123,"fill":99,"stroke":158,"style":124},[101,173,174],{"x":153,"y":145,"fill":105,"style":128},"Request C runs too",[101,176,177],{"x":153,"y":149,"fill":138,"style":128},"pool limit calls in flight",[42,179],{},[45,181,183],{"id":182},"why-requests-freezes-a-fastapi-worker","Why requests freezes a FastAPI worker",[14,185,186,187,190,191,195,196,199],{},"This is the trap that bites builders new to async. Inside an ",[18,188,189],{},"async def"," handler in ",[26,192,194],{"href":193},"\u002Fgetting-started-with-python-apis-for-builders\u002Fsetting-up-fastapi\u002F","FastAPI",", the event loop drives every connection on a single thread. When you call ",[18,197,198],{},"requests.get(...)"," there, it makes a blocking socket read and never yields. While it waits on the upstream, the whole loop is frozen: no other request in that worker progresses, not even the health check, not even requests that touch nothing but a local cache.",[201,202,207],"pre",{"className":203,"code":204,"language":205,"meta":206,"style":206},"language-python shiki shiki-themes github-light github-dark","# WRONG: blocks the whole event loop while it waits\nimport os\nimport requests\nfrom fastapi import FastAPI\n\nUPSTREAM_URL = os.getenv(\"UPSTREAM_URL\", \"https:\u002F\u002Fexample.test\u002Fdata\")\napp = FastAPI()\n\n\n@app.get(\"\u002Fbad\")\nasync def bad():\n    # This synchronous call stalls every other request in this worker\n    resp = requests.get(UPSTREAM_URL, timeout=10)\n    return resp.json()\n","python","",[18,208,209,218,229,237,251,258,284,296,301,306,321,336,342,368],{"__ignoreMap":206},[210,211,214],"span",{"class":212,"line":213},"line",1,[210,215,217],{"class":216},"sJ8bj","# WRONG: blocks the whole event loop while it waits\n",[210,219,221,225],{"class":212,"line":220},2,[210,222,224],{"class":223},"szBVR","import",[210,226,228],{"class":227},"sVt8B"," os\n",[210,230,232,234],{"class":212,"line":231},3,[210,233,224],{"class":223},[210,235,236],{"class":227}," requests\n",[210,238,240,243,246,248],{"class":212,"line":239},4,[210,241,242],{"class":223},"from",[210,244,245],{"class":227}," fastapi ",[210,247,224],{"class":223},[210,249,250],{"class":227}," FastAPI\n",[210,252,254],{"class":212,"line":253},5,[210,255,257],{"emptyLinePlaceholder":256},true,"\n",[210,259,261,265,268,271,275,278,281],{"class":212,"line":260},6,[210,262,264],{"class":263},"sj4cs","UPSTREAM_URL",[210,266,267],{"class":223}," =",[210,269,270],{"class":227}," os.getenv(",[210,272,274],{"class":273},"sZZnC","\"UPSTREAM_URL\"",[210,276,277],{"class":227},", ",[210,279,280],{"class":273},"\"https:\u002F\u002Fexample.test\u002Fdata\"",[210,282,283],{"class":227},")\n",[210,285,287,290,293],{"class":212,"line":286},7,[210,288,289],{"class":227},"app ",[210,291,292],{"class":223},"=",[210,294,295],{"class":227}," FastAPI()\n",[210,297,299],{"class":212,"line":298},8,[210,300,257],{"emptyLinePlaceholder":256},[210,302,304],{"class":212,"line":303},9,[210,305,257],{"emptyLinePlaceholder":256},[210,307,309,313,316,319],{"class":212,"line":308},10,[210,310,312],{"class":311},"sScJk","@app.get",[210,314,315],{"class":227},"(",[210,317,318],{"class":273},"\"\u002Fbad\"",[210,320,283],{"class":227},[210,322,324,327,330,333],{"class":212,"line":323},11,[210,325,326],{"class":223},"async",[210,328,329],{"class":223}," def",[210,331,332],{"class":311}," bad",[210,334,335],{"class":227},"():\n",[210,337,339],{"class":212,"line":338},12,[210,340,341],{"class":216},"    # This synchronous call stalls every other request in this worker\n",[210,343,345,348,350,353,355,357,361,363,366],{"class":212,"line":344},13,[210,346,347],{"class":227},"    resp ",[210,349,292],{"class":223},[210,351,352],{"class":227}," requests.get(",[210,354,264],{"class":263},[210,356,277],{"class":227},[210,358,360],{"class":359},"s4XuR","timeout",[210,362,292],{"class":223},[210,364,365],{"class":263},"10",[210,367,283],{"class":227},[210,369,371,374],{"class":212,"line":370},14,[210,372,373],{"class":223},"    return",[210,375,376],{"class":227}," resp.json()\n",[14,378,379,380,383,384,386],{},"The numbers make it concrete. Take a 120 ms upstream call, the sort of latency a payment or enrichment API gives you on a good day. A worker that blocks on it serves at most 1000 \u002F 120 ≈ 8.3 requests per second, no matter how much CPU you bought. Move the same call to a ",[18,381,382],{},"def"," handler and FastAPI hands it to the threadpool, which lifts you to roughly the thread count — around 330 requests per second for the default 40 threads, minus the memory and context-switch tax of the threads themselves. Await it through ",[18,385,36],{}," with a 100-connection pool and the same worker holds 100 calls in flight, a ceiling near 830 requests per second before anything else becomes the bottleneck.",[75,388,84,393,84,396,84,399,84,402,84,408,84,413,84,419,84,424,84,430,84,434,84,439,84,444,84,448,84,452,84,457,84,461,84,464,84,466],{"viewBox":389,"role":78,"ariaLabelledBy":390,"xmlns":82,"style":83},"0 0 720 270",[391,392],"hvr-tput-t","hvr-tput-d",[86,394,395],{"id":391},"Throughput per worker for a 120 millisecond upstream call",[90,397,398],{"id":392},"Horizontal bars: blocking requests in an async handler reaches 8 requests per second, requests in a threadpool handler reaches 333, and awaited httpx reaches 833.",[94,400],{"x":96,"y":96,"width":97,"height":401,"fill":99},"270",[101,403,407],{"x":404,"y":405,"fill":105,"style":406},"16","28","font-size:14;font-family:var(--font-sans);","Requests per second, one worker, 120 ms upstream",[212,409],{"x1":410,"y1":411,"x2":410,"y2":412,"stroke":133,"style":117},"200","52","212",[101,414,418],{"x":415,"y":416,"fill":105,"style":417},"188","82","text-anchor:end;font-size:12;font-family:var(--font-sans);","requests, async def",[94,420],{"x":410,"y":120,"width":421,"height":422,"rx":423,"fill":116},"5","34","2",[101,425,429],{"x":426,"y":427,"fill":105,"style":428},"215","86","font-size:12;font-family:var(--font-sans);","8 rps — loop blocked",[101,431,433],{"x":415,"y":432,"fill":105,"style":417},"142","requests, sync def",[94,435],{"x":410,"y":436,"width":437,"height":422,"rx":423,"fill":438},"124","184","var(--c-yellow)",[101,440,443],{"x":441,"y":442,"fill":105,"style":428},"394","146","333 rps — 40 threads",[101,445,447],{"x":415,"y":446,"fill":105,"style":417},"202","httpx AsyncClient",[94,449],{"x":410,"y":437,"width":450,"height":422,"rx":423,"fill":158,"fillOpacity":451,"stroke":158,"style":124},"460",0.22,[101,453,456],{"x":454,"y":455,"fill":105,"style":128},"430","206","833 rps — 100 pooled connections",[212,458],{"x1":410,"y1":459,"x2":460,"y2":459,"stroke":133,"style":124},"228","660",[101,462,96],{"x":410,"y":463,"fill":138,"style":128},"248",[101,465,157],{"x":454,"y":463,"fill":138,"style":128},[101,467,468],{"x":460,"y":463,"fill":138,"style":128},"800",[14,470,471,472,474,475,478],{},"If you are stuck with a blocking client — an SDK that wraps ",[18,473,20],{}," and exposes no async surface — do not call it inline. Push it to a thread with ",[18,476,477],{},"anyio.to_thread.run_sync"," so the loop keeps turning, and treat the thread count as a hard concurrency limit you now have to monitor. That is a workaround, not a design.",[42,480],{},[45,482,484],{"id":483},"the-same-call-in-three-clients","The same call in three clients",[14,486,487,488,490,491,493,494,496],{},"Here is one GET expressed three ways: synchronous ",[18,489,20],{},", synchronous ",[18,492,58],{},", and async ",[18,495,58],{},". The sync versions are nearly identical, which is exactly why the migration is cheap.",[201,498,500],{"className":203,"code":499,"language":205,"meta":206,"style":206},"import os\nimport asyncio\nimport requests\nimport httpx\n\nUPSTREAM_URL = os.getenv(\"UPSTREAM_URL\", \"https:\u002F\u002Fexample.test\u002Fdata\")\nTIMEOUT = float(os.getenv(\"HTTP_TIMEOUT\", \"10\"))\n\n\n# 1. Synchronous requests — fine in scripts and threaded servers\ndef fetch_requests() -> dict:\n    resp = requests.get(UPSTREAM_URL, timeout=TIMEOUT)\n    resp.raise_for_status()\n    return resp.json()\n\n\n# 2. Synchronous httpx — same shape, drop-in replacement\ndef fetch_httpx_sync() -> dict:\n    with httpx.Client(timeout=TIMEOUT) as client:\n        resp = client.get(UPSTREAM_URL)\n        resp.raise_for_status()\n        return resp.json()\n\n\n# 3. Async httpx — non-blocking, the right tool inside an event loop\nasync def fetch_httpx_async() -> dict:\n    async with httpx.AsyncClient(timeout=TIMEOUT) as client:\n        resp = await client.get(UPSTREAM_URL)\n        resp.raise_for_status()\n        return resp.json()\n\n\n# Concurrency is where async pays off: these calls overlap instead of queueing\nasync def fetch_many(client: httpx.AsyncClient, urls: list[str]) -> list[httpx.Response]:\n    sem = asyncio.Semaphore(int(os.getenv(\"HTTP_FANOUT\", \"20\")))\n\n    async def one(url: str) -> httpx.Response:\n        async with sem:\n            return await client.get(url)\n\n    return await asyncio.gather(*(one(u) for u in urls))\n",[18,501,502,508,515,521,528,532,548,572,576,580,585,601,621,626,632,637,642,648,662,686,701,707,715,720,725,731,747,771,787,792,799,804,809,815,834,861,866,884,895,906,911],{"__ignoreMap":206},[210,503,504,506],{"class":212,"line":213},[210,505,224],{"class":223},[210,507,228],{"class":227},[210,509,510,512],{"class":212,"line":220},[210,511,224],{"class":223},[210,513,514],{"class":227}," asyncio\n",[210,516,517,519],{"class":212,"line":231},[210,518,224],{"class":223},[210,520,236],{"class":227},[210,522,523,525],{"class":212,"line":239},[210,524,224],{"class":223},[210,526,527],{"class":227}," httpx\n",[210,529,530],{"class":212,"line":253},[210,531,257],{"emptyLinePlaceholder":256},[210,533,534,536,538,540,542,544,546],{"class":212,"line":260},[210,535,264],{"class":263},[210,537,267],{"class":223},[210,539,270],{"class":227},[210,541,274],{"class":273},[210,543,277],{"class":227},[210,545,280],{"class":273},[210,547,283],{"class":227},[210,549,550,553,555,558,561,564,566,569],{"class":212,"line":286},[210,551,552],{"class":263},"TIMEOUT",[210,554,267],{"class":223},[210,556,557],{"class":263}," float",[210,559,560],{"class":227},"(os.getenv(",[210,562,563],{"class":273},"\"HTTP_TIMEOUT\"",[210,565,277],{"class":227},[210,567,568],{"class":273},"\"10\"",[210,570,571],{"class":227},"))\n",[210,573,574],{"class":212,"line":298},[210,575,257],{"emptyLinePlaceholder":256},[210,577,578],{"class":212,"line":303},[210,579,257],{"emptyLinePlaceholder":256},[210,581,582],{"class":212,"line":308},[210,583,584],{"class":216},"# 1. Synchronous requests — fine in scripts and threaded servers\n",[210,586,587,589,592,595,598],{"class":212,"line":323},[210,588,382],{"class":223},[210,590,591],{"class":311}," fetch_requests",[210,593,594],{"class":227},"() -> ",[210,596,597],{"class":263},"dict",[210,599,600],{"class":227},":\n",[210,602,603,605,607,609,611,613,615,617,619],{"class":212,"line":338},[210,604,347],{"class":227},[210,606,292],{"class":223},[210,608,352],{"class":227},[210,610,264],{"class":263},[210,612,277],{"class":227},[210,614,360],{"class":359},[210,616,292],{"class":223},[210,618,552],{"class":263},[210,620,283],{"class":227},[210,622,623],{"class":212,"line":344},[210,624,625],{"class":227},"    resp.raise_for_status()\n",[210,627,628,630],{"class":212,"line":370},[210,629,373],{"class":223},[210,631,376],{"class":227},[210,633,635],{"class":212,"line":634},15,[210,636,257],{"emptyLinePlaceholder":256},[210,638,640],{"class":212,"line":639},16,[210,641,257],{"emptyLinePlaceholder":256},[210,643,645],{"class":212,"line":644},17,[210,646,647],{"class":216},"# 2. Synchronous httpx — same shape, drop-in replacement\n",[210,649,651,653,656,658,660],{"class":212,"line":650},18,[210,652,382],{"class":223},[210,654,655],{"class":311}," fetch_httpx_sync",[210,657,594],{"class":227},[210,659,597],{"class":263},[210,661,600],{"class":227},[210,663,665,668,671,673,675,677,680,683],{"class":212,"line":664},19,[210,666,667],{"class":223},"    with",[210,669,670],{"class":227}," httpx.Client(",[210,672,360],{"class":359},[210,674,292],{"class":223},[210,676,552],{"class":263},[210,678,679],{"class":227},") ",[210,681,682],{"class":223},"as",[210,684,685],{"class":227}," client:\n",[210,687,689,692,694,697,699],{"class":212,"line":688},20,[210,690,691],{"class":227},"        resp ",[210,693,292],{"class":223},[210,695,696],{"class":227}," client.get(",[210,698,264],{"class":263},[210,700,283],{"class":227},[210,702,704],{"class":212,"line":703},21,[210,705,706],{"class":227},"        resp.raise_for_status()\n",[210,708,710,713],{"class":212,"line":709},22,[210,711,712],{"class":223},"        return",[210,714,376],{"class":227},[210,716,718],{"class":212,"line":717},23,[210,719,257],{"emptyLinePlaceholder":256},[210,721,723],{"class":212,"line":722},24,[210,724,257],{"emptyLinePlaceholder":256},[210,726,728],{"class":212,"line":727},25,[210,729,730],{"class":216},"# 3. Async httpx — non-blocking, the right tool inside an event loop\n",[210,732,734,736,738,741,743,745],{"class":212,"line":733},26,[210,735,326],{"class":223},[210,737,329],{"class":223},[210,739,740],{"class":311}," fetch_httpx_async",[210,742,594],{"class":227},[210,744,597],{"class":263},[210,746,600],{"class":227},[210,748,750,753,756,759,761,763,765,767,769],{"class":212,"line":749},27,[210,751,752],{"class":223},"    async",[210,754,755],{"class":223}," with",[210,757,758],{"class":227}," httpx.AsyncClient(",[210,760,360],{"class":359},[210,762,292],{"class":223},[210,764,552],{"class":263},[210,766,679],{"class":227},[210,768,682],{"class":223},[210,770,685],{"class":227},[210,772,774,776,778,781,783,785],{"class":212,"line":773},28,[210,775,691],{"class":227},[210,777,292],{"class":223},[210,779,780],{"class":223}," await",[210,782,696],{"class":227},[210,784,264],{"class":263},[210,786,283],{"class":227},[210,788,790],{"class":212,"line":789},29,[210,791,706],{"class":227},[210,793,795,797],{"class":212,"line":794},30,[210,796,712],{"class":223},[210,798,376],{"class":227},[210,800,802],{"class":212,"line":801},31,[210,803,257],{"emptyLinePlaceholder":256},[210,805,807],{"class":212,"line":806},32,[210,808,257],{"emptyLinePlaceholder":256},[210,810,812],{"class":212,"line":811},33,[210,813,814],{"class":216},"# Concurrency is where async pays off: these calls overlap instead of queueing\n",[210,816,818,820,822,825,828,831],{"class":212,"line":817},34,[210,819,326],{"class":223},[210,821,329],{"class":223},[210,823,824],{"class":311}," fetch_many",[210,826,827],{"class":227},"(client: httpx.AsyncClient, urls: list[",[210,829,830],{"class":263},"str",[210,832,833],{"class":227},"]) -> list[httpx.Response]:\n",[210,835,837,840,842,845,848,850,853,855,858],{"class":212,"line":836},35,[210,838,839],{"class":227},"    sem ",[210,841,292],{"class":223},[210,843,844],{"class":227}," asyncio.Semaphore(",[210,846,847],{"class":263},"int",[210,849,560],{"class":227},[210,851,852],{"class":273},"\"HTTP_FANOUT\"",[210,854,277],{"class":227},[210,856,857],{"class":273},"\"20\"",[210,859,860],{"class":227},")))\n",[210,862,864],{"class":212,"line":863},36,[210,865,257],{"emptyLinePlaceholder":256},[210,867,869,871,873,876,879,881],{"class":212,"line":868},37,[210,870,752],{"class":223},[210,872,329],{"class":223},[210,874,875],{"class":311}," one",[210,877,878],{"class":227},"(url: ",[210,880,830],{"class":263},[210,882,883],{"class":227},") -> httpx.Response:\n",[210,885,887,890,892],{"class":212,"line":886},38,[210,888,889],{"class":223},"        async",[210,891,755],{"class":223},[210,893,894],{"class":227}," sem:\n",[210,896,898,901,903],{"class":212,"line":897},39,[210,899,900],{"class":223},"            return",[210,902,780],{"class":223},[210,904,905],{"class":227}," client.get(url)\n",[210,907,909],{"class":212,"line":908},40,[210,910,257],{"emptyLinePlaceholder":256},[210,912,914,916,918,921,924,927,930,933,936],{"class":212,"line":913},41,[210,915,373],{"class":223},[210,917,780],{"class":223},[210,919,920],{"class":227}," asyncio.gather(",[210,922,923],{"class":223},"*",[210,925,926],{"class":227},"(one(u) ",[210,928,929],{"class":223},"for",[210,931,932],{"class":227}," u ",[210,934,935],{"class":223},"in",[210,937,938],{"class":227}," urls))\n",[14,940,941,942,945,946,949,950,953],{},"Note the semaphore in ",[18,943,944],{},"fetch_many",". A bare ",[18,947,948],{},"asyncio.gather"," over a list you do not control is how a fan-out becomes an accidental denial-of-service against your own supplier — the pool queues the excess, every queued call still counts against its pool timeout, and you get a burst of ",[18,951,952],{},"PoolTimeout"," errors that look like an upstream outage but are entirely self-inflicted. Bound the fan-out explicitly and the failure mode disappears.",[42,955],{},[45,957,959],{"id":958},"timeouts-and-pools-the-settings-that-actually-matter","Timeouts and pools: the settings that actually matter",[14,961,962,963,965,966,969,970,973,974,976,977,980],{},"Both libraries pool connections, but the defaults pull in opposite directions. With ",[18,964,20],{}," you must create a ",[18,967,968],{},"Session"," to reuse TCP connections; a bare ",[18,971,972],{},"requests.get"," opens a fresh connection every time. ",[18,975,58],{}," pools automatically whenever you reuse a client instance, and exposes the pool through ",[18,978,979],{},"httpx.Limits",".",[14,982,983,984,986,987,991,992,994,995,999],{},"The bigger edge is timeouts. A bare ",[18,985,972],{}," has ",[988,989,990],"strong",{},"no timeout at all"," by default, so one hung upstream can pin a worker until the process is restarted — still one of the most common causes of a small API going dark. ",[18,993,58],{}," ships a 5-second default and splits it into four independent phases: connect, write, read, and pool acquisition. Independent is the word to internalise. The phases do not share a budget, so a client configured with connect 5, write 10, read 10 and pool 5 can legitimately spend 30 seconds on a single attempt, and if you then wrap it in ",[26,996,998],{"href":997},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fretrying-failed-http-requests-with-tenacity\u002F","retries with tenacity"," you have quietly signed up for a two-minute worst case on a request your customer expects in under a second.",[201,1001,1003],{"className":203,"code":1002,"language":205,"meta":206,"style":206},"import os\nimport httpx\n\nCONNECT_T = float(os.getenv(\"HTTP_CONNECT_TIMEOUT\", \"2.0\"))\nREAD_T = float(os.getenv(\"HTTP_READ_TIMEOUT\", \"4.0\"))\nWRITE_T = float(os.getenv(\"HTTP_WRITE_TIMEOUT\", \"2.0\"))\nPOOL_T = float(os.getenv(\"HTTP_POOL_TIMEOUT\", \"1.0\"))\nMAX_CONN = int(os.getenv(\"HTTP_MAX_CONNECTIONS\", \"100\"))\nMAX_KEEPALIVE = int(os.getenv(\"HTTP_MAX_KEEPALIVE\", \"20\"))\n\ntimeout = httpx.Timeout(connect=CONNECT_T, read=READ_T, write=WRITE_T, pool=POOL_T)\nlimits = httpx.Limits(\n    max_connections=MAX_CONN,\n    max_keepalive_connections=MAX_KEEPALIVE,\n    keepalive_expiry=float(os.getenv(\"HTTP_KEEPALIVE_EXPIRY\", \"20\")),\n)\n",[18,1004,1005,1011,1017,1021,1042,1063,1083,1104,1126,1146,1150,1196,1206,1218,1229,1251],{"__ignoreMap":206},[210,1006,1007,1009],{"class":212,"line":213},[210,1008,224],{"class":223},[210,1010,228],{"class":227},[210,1012,1013,1015],{"class":212,"line":220},[210,1014,224],{"class":223},[210,1016,527],{"class":227},[210,1018,1019],{"class":212,"line":231},[210,1020,257],{"emptyLinePlaceholder":256},[210,1022,1023,1026,1028,1030,1032,1035,1037,1040],{"class":212,"line":239},[210,1024,1025],{"class":263},"CONNECT_T",[210,1027,267],{"class":223},[210,1029,557],{"class":263},[210,1031,560],{"class":227},[210,1033,1034],{"class":273},"\"HTTP_CONNECT_TIMEOUT\"",[210,1036,277],{"class":227},[210,1038,1039],{"class":273},"\"2.0\"",[210,1041,571],{"class":227},[210,1043,1044,1047,1049,1051,1053,1056,1058,1061],{"class":212,"line":253},[210,1045,1046],{"class":263},"READ_T",[210,1048,267],{"class":223},[210,1050,557],{"class":263},[210,1052,560],{"class":227},[210,1054,1055],{"class":273},"\"HTTP_READ_TIMEOUT\"",[210,1057,277],{"class":227},[210,1059,1060],{"class":273},"\"4.0\"",[210,1062,571],{"class":227},[210,1064,1065,1068,1070,1072,1074,1077,1079,1081],{"class":212,"line":260},[210,1066,1067],{"class":263},"WRITE_T",[210,1069,267],{"class":223},[210,1071,557],{"class":263},[210,1073,560],{"class":227},[210,1075,1076],{"class":273},"\"HTTP_WRITE_TIMEOUT\"",[210,1078,277],{"class":227},[210,1080,1039],{"class":273},[210,1082,571],{"class":227},[210,1084,1085,1088,1090,1092,1094,1097,1099,1102],{"class":212,"line":286},[210,1086,1087],{"class":263},"POOL_T",[210,1089,267],{"class":223},[210,1091,557],{"class":263},[210,1093,560],{"class":227},[210,1095,1096],{"class":273},"\"HTTP_POOL_TIMEOUT\"",[210,1098,277],{"class":227},[210,1100,1101],{"class":273},"\"1.0\"",[210,1103,571],{"class":227},[210,1105,1106,1109,1111,1114,1116,1119,1121,1124],{"class":212,"line":298},[210,1107,1108],{"class":263},"MAX_CONN",[210,1110,267],{"class":223},[210,1112,1113],{"class":263}," int",[210,1115,560],{"class":227},[210,1117,1118],{"class":273},"\"HTTP_MAX_CONNECTIONS\"",[210,1120,277],{"class":227},[210,1122,1123],{"class":273},"\"100\"",[210,1125,571],{"class":227},[210,1127,1128,1131,1133,1135,1137,1140,1142,1144],{"class":212,"line":303},[210,1129,1130],{"class":263},"MAX_KEEPALIVE",[210,1132,267],{"class":223},[210,1134,1113],{"class":263},[210,1136,560],{"class":227},[210,1138,1139],{"class":273},"\"HTTP_MAX_KEEPALIVE\"",[210,1141,277],{"class":227},[210,1143,857],{"class":273},[210,1145,571],{"class":227},[210,1147,1148],{"class":212,"line":308},[210,1149,257],{"emptyLinePlaceholder":256},[210,1151,1152,1155,1157,1160,1163,1165,1167,1169,1172,1174,1176,1178,1181,1183,1185,1187,1190,1192,1194],{"class":212,"line":323},[210,1153,1154],{"class":227},"timeout ",[210,1156,292],{"class":223},[210,1158,1159],{"class":227}," httpx.Timeout(",[210,1161,1162],{"class":359},"connect",[210,1164,292],{"class":223},[210,1166,1025],{"class":263},[210,1168,277],{"class":227},[210,1170,1171],{"class":359},"read",[210,1173,292],{"class":223},[210,1175,1046],{"class":263},[210,1177,277],{"class":227},[210,1179,1180],{"class":359},"write",[210,1182,292],{"class":223},[210,1184,1067],{"class":263},[210,1186,277],{"class":227},[210,1188,1189],{"class":359},"pool",[210,1191,292],{"class":223},[210,1193,1087],{"class":263},[210,1195,283],{"class":227},[210,1197,1198,1201,1203],{"class":212,"line":338},[210,1199,1200],{"class":227},"limits ",[210,1202,292],{"class":223},[210,1204,1205],{"class":227}," httpx.Limits(\n",[210,1207,1208,1211,1213,1215],{"class":212,"line":344},[210,1209,1210],{"class":359},"    max_connections",[210,1212,292],{"class":223},[210,1214,1108],{"class":263},[210,1216,1217],{"class":227},",\n",[210,1219,1220,1223,1225,1227],{"class":212,"line":370},[210,1221,1222],{"class":359},"    max_keepalive_connections",[210,1224,292],{"class":223},[210,1226,1130],{"class":263},[210,1228,1217],{"class":227},[210,1230,1231,1234,1236,1239,1241,1244,1246,1248],{"class":212,"line":634},[210,1232,1233],{"class":359},"    keepalive_expiry",[210,1235,292],{"class":223},[210,1237,1238],{"class":263},"float",[210,1240,560],{"class":227},[210,1242,1243],{"class":273},"\"HTTP_KEEPALIVE_EXPIRY\"",[210,1245,277],{"class":227},[210,1247,857],{"class":273},[210,1249,1250],{"class":227},")),\n",[210,1252,1253],{"class":212,"line":639},[210,1254,283],{"class":227},[14,1256,1257,1258,1262,1263,1265,1266,1269,1270,1273,1274,1276],{},"Set the pool timeout low and deliberately. It is the only one of the four that reports a problem on ",[1259,1260,1261],"em",{},"your"," side of the wire: a ",[18,1264,952],{}," means your own concurrency exceeded ",[18,1267,1268],{},"max_connections",", and a one-second ceiling turns that into a fast, obvious error instead of a slow mystery. Keep ",[18,1271,1272],{},"max_keepalive_connections"," well below ",[18,1275,1268],{}," too, because idle sockets held open against an upstream that recycles them produce sporadic read errors on the first byte of a reused connection.",[75,1278,84,1283,84,1286,84,1289,84,1292,84,1295,84,1300,84,1305,84,1308,84,1311,84,1315,84,1319,84,1322,84,1326,84,1331,84,1335,84,1340,84,1344,84,1347,84,1350,84,1354,84,1357,84,1361,84,1365,84,1368,84,1371],{"viewBox":1279,"role":78,"ariaLabelledBy":1280,"xmlns":82,"style":83},"0 0 720 290",[1281,1282],"hvr-to-t","hvr-to-d",[86,1284,1285],{"id":1281},"Per-phase httpx timeouts add up to the real ceiling",[90,1287,1288],{"id":1282},"A timeline comparing loose per-phase timeouts totalling thirty seconds per attempt with a budgeted configuration totalling three and a half seconds.",[94,1290],{"x":96,"y":96,"width":97,"height":1291,"fill":99},"290",[101,1293,1294],{"x":110,"y":104,"fill":105,"style":406},"Loose phases: pool 5 + connect 5 + write 10 + read 10",[94,1296],{"x":110,"y":1297,"width":1298,"height":110,"rx":1299,"fill":115,"stroke":116,"style":124},"44","107","4",[101,1301,1304],{"x":1302,"y":1303,"fill":105,"style":128},"93","69","pool 5s",[94,1306],{"x":1307,"y":1297,"width":1298,"height":110,"rx":1299,"fill":115,"stroke":116,"style":124},"147",[101,1309,1310],{"x":410,"y":1303,"fill":105,"style":128},"connect 5s",[94,1312],{"x":1313,"y":1297,"width":1314,"height":110,"rx":1299,"fill":115,"stroke":116,"style":124},"254","213",[101,1316,1318],{"x":1317,"y":1303,"fill":105,"style":128},"360","write 10s",[94,1320],{"x":1321,"y":1297,"width":1314,"height":110,"rx":1299,"fill":115,"stroke":116,"style":124},"467",[101,1323,1325],{"x":1324,"y":1303,"fill":105,"style":128},"573","read 10s",[101,1327,1330],{"x":1328,"y":1329,"fill":138,"style":417},"680","106","30 s worst case per attempt",[101,1332,1334],{"x":110,"y":1333,"fill":105,"style":406},"152","Budgeted for a 3 s customer promise",[94,1336],{"x":110,"y":1337,"width":1338,"height":110,"rx":1339,"fill":115,"stroke":158,"style":124},"166","11","3",[94,1341],{"x":1342,"y":1337,"width":1343,"height":110,"rx":1339,"fill":115,"stroke":158,"style":124},"51","43",[94,1345],{"x":1346,"y":1337,"width":1343,"height":110,"rx":1339,"fill":115,"stroke":158,"style":124},"94",[94,1348],{"x":1349,"y":1337,"width":120,"height":110,"rx":1339,"fill":115,"stroke":158,"style":124},"137",[101,1351,1353],{"x":1314,"y":1352,"fill":105,"style":428},"191","3.5 s total: 0.5 \u002F 1 \u002F 0.5 \u002F 1.5",[212,1355],{"x1":110,"y1":1356,"x2":1328,"y2":1356,"stroke":133,"style":124},"230",[101,1358,1360],{"x":110,"y":1359,"fill":138,"style":128},"250","0s",[101,1362,1364],{"x":1363,"y":1359,"fill":138,"style":128},"253","10s",[101,1366,1367],{"x":1321,"y":1359,"fill":138,"style":128},"20s",[101,1369,1370],{"x":1328,"y":1359,"fill":138,"style":128},"30s",[101,1372,1374],{"x":110,"y":1373,"fill":138,"style":428},"278","Each phase has its own timer — the ceiling is their sum, not the largest one.",[42,1376],{},[45,1378,1380],{"id":1379},"client-lifetime-the-mistake-that-costs-90-ms-per-call","Client lifetime: the mistake that costs 90 ms per call",[14,1382,1383,1384,1386,1387,1390,1391,1394],{},"Creating a client inside the handler is the single most common way builders throw away everything ",[18,1385,58],{}," just gave them. Every ",[18,1388,1389],{},"async with httpx.AsyncClient() as client:"," inside a route means a fresh DNS lookup, TCP handshake and TLS negotiation on every request — call it 90 ms against a TLS endpoint on another continent, on top of the 120 ms the upstream actually needs. Worse, it also destroys the pool limits, because ten concurrent handlers now hold ten independent pools and your ",[18,1392,1393],{},"max_connections=100"," has quietly become 1,000 as far as the upstream is concerned.",[14,1396,1397],{},"Build the client once in the lifespan and hang it off application state.",[201,1399,1401],{"className":203,"code":1400,"language":205,"meta":206,"style":206},"import os\nfrom contextlib import asynccontextmanager\n\nimport httpx\nfrom fastapi import FastAPI, Request\n\nBASE_URL = os.getenv(\"UPSTREAM_BASE_URL\", \"https:\u002F\u002Fexample.test\")\nAPI_KEY = os.environ[\"UPSTREAM_API_KEY\"]\n\n\n@asynccontextmanager\nasync def lifespan(app: FastAPI):\n    async with httpx.AsyncClient(\n        base_url=BASE_URL,\n        headers={\"Authorization\": f\"Bearer {API_KEY}\"},\n        timeout=httpx.Timeout(connect=2.0, read=4.0, write=2.0, pool=1.0),\n        limits=httpx.Limits(max_connections=100, max_keepalive_connections=20),\n        http2=os.getenv(\"HTTP2_ENABLED\", \"false\").lower() == \"true\",\n    ) as client:\n        app.state.http = client\n        yield\n\n\napp = FastAPI(lifespan=lifespan)\n\n\n@app.get(\"\u002Fitems\u002F{item_id}\")\nasync def get_item(item_id: str, request: Request):\n    client: httpx.AsyncClient = request.app.state.http\n    resp = await client.get(f\"\u002Fitems\u002F{item_id}\")\n    resp.raise_for_status()\n    return resp.json()\n",[18,1402,1403,1409,1421,1425,1431,1442,1446,1465,1481,1485,1489,1494,1506,1515,1526,1557,1603,1631,1660,1669,1679,1684,1688,1692,1709,1713,1717,1733,1750,1760,1786,1790],{"__ignoreMap":206},[210,1404,1405,1407],{"class":212,"line":213},[210,1406,224],{"class":223},[210,1408,228],{"class":227},[210,1410,1411,1413,1416,1418],{"class":212,"line":220},[210,1412,242],{"class":223},[210,1414,1415],{"class":227}," contextlib ",[210,1417,224],{"class":223},[210,1419,1420],{"class":227}," asynccontextmanager\n",[210,1422,1423],{"class":212,"line":231},[210,1424,257],{"emptyLinePlaceholder":256},[210,1426,1427,1429],{"class":212,"line":239},[210,1428,224],{"class":223},[210,1430,527],{"class":227},[210,1432,1433,1435,1437,1439],{"class":212,"line":253},[210,1434,242],{"class":223},[210,1436,245],{"class":227},[210,1438,224],{"class":223},[210,1440,1441],{"class":227}," FastAPI, Request\n",[210,1443,1444],{"class":212,"line":260},[210,1445,257],{"emptyLinePlaceholder":256},[210,1447,1448,1451,1453,1455,1458,1460,1463],{"class":212,"line":286},[210,1449,1450],{"class":263},"BASE_URL",[210,1452,267],{"class":223},[210,1454,270],{"class":227},[210,1456,1457],{"class":273},"\"UPSTREAM_BASE_URL\"",[210,1459,277],{"class":227},[210,1461,1462],{"class":273},"\"https:\u002F\u002Fexample.test\"",[210,1464,283],{"class":227},[210,1466,1467,1470,1472,1475,1478],{"class":212,"line":298},[210,1468,1469],{"class":263},"API_KEY",[210,1471,267],{"class":223},[210,1473,1474],{"class":227}," os.environ[",[210,1476,1477],{"class":273},"\"UPSTREAM_API_KEY\"",[210,1479,1480],{"class":227},"]\n",[210,1482,1483],{"class":212,"line":303},[210,1484,257],{"emptyLinePlaceholder":256},[210,1486,1487],{"class":212,"line":308},[210,1488,257],{"emptyLinePlaceholder":256},[210,1490,1491],{"class":212,"line":323},[210,1492,1493],{"class":311},"@asynccontextmanager\n",[210,1495,1496,1498,1500,1503],{"class":212,"line":338},[210,1497,326],{"class":223},[210,1499,329],{"class":223},[210,1501,1502],{"class":311}," lifespan",[210,1504,1505],{"class":227},"(app: FastAPI):\n",[210,1507,1508,1510,1512],{"class":212,"line":344},[210,1509,752],{"class":223},[210,1511,755],{"class":223},[210,1513,1514],{"class":227}," httpx.AsyncClient(\n",[210,1516,1517,1520,1522,1524],{"class":212,"line":370},[210,1518,1519],{"class":359},"        base_url",[210,1521,292],{"class":223},[210,1523,1450],{"class":263},[210,1525,1217],{"class":227},[210,1527,1528,1531,1533,1536,1539,1542,1545,1548,1551,1554],{"class":212,"line":634},[210,1529,1530],{"class":359},"        headers",[210,1532,292],{"class":223},[210,1534,1535],{"class":227},"{",[210,1537,1538],{"class":273},"\"Authorization\"",[210,1540,1541],{"class":227},": ",[210,1543,1544],{"class":223},"f",[210,1546,1547],{"class":273},"\"Bearer ",[210,1549,1550],{"class":263},"{API_KEY}",[210,1552,1553],{"class":273},"\"",[210,1555,1556],{"class":227},"},\n",[210,1558,1559,1562,1564,1567,1569,1571,1574,1576,1578,1580,1583,1585,1587,1589,1591,1593,1595,1597,1600],{"class":212,"line":639},[210,1560,1561],{"class":359},"        timeout",[210,1563,292],{"class":223},[210,1565,1566],{"class":227},"httpx.Timeout(",[210,1568,1162],{"class":359},[210,1570,292],{"class":223},[210,1572,1573],{"class":263},"2.0",[210,1575,277],{"class":227},[210,1577,1171],{"class":359},[210,1579,292],{"class":223},[210,1581,1582],{"class":263},"4.0",[210,1584,277],{"class":227},[210,1586,1180],{"class":359},[210,1588,292],{"class":223},[210,1590,1573],{"class":263},[210,1592,277],{"class":227},[210,1594,1189],{"class":359},[210,1596,292],{"class":223},[210,1598,1599],{"class":263},"1.0",[210,1601,1602],{"class":227},"),\n",[210,1604,1605,1608,1610,1613,1615,1617,1620,1622,1624,1626,1629],{"class":212,"line":644},[210,1606,1607],{"class":359},"        limits",[210,1609,292],{"class":223},[210,1611,1612],{"class":227},"httpx.Limits(",[210,1614,1268],{"class":359},[210,1616,292],{"class":223},[210,1618,1619],{"class":263},"100",[210,1621,277],{"class":227},[210,1623,1272],{"class":359},[210,1625,292],{"class":223},[210,1627,1628],{"class":263},"20",[210,1630,1602],{"class":227},[210,1632,1633,1636,1638,1641,1644,1646,1649,1652,1655,1658],{"class":212,"line":650},[210,1634,1635],{"class":359},"        http2",[210,1637,292],{"class":223},[210,1639,1640],{"class":227},"os.getenv(",[210,1642,1643],{"class":273},"\"HTTP2_ENABLED\"",[210,1645,277],{"class":227},[210,1647,1648],{"class":273},"\"false\"",[210,1650,1651],{"class":227},").lower() ",[210,1653,1654],{"class":223},"==",[210,1656,1657],{"class":273}," \"true\"",[210,1659,1217],{"class":227},[210,1661,1662,1665,1667],{"class":212,"line":664},[210,1663,1664],{"class":227},"    ) ",[210,1666,682],{"class":223},[210,1668,685],{"class":227},[210,1670,1671,1674,1676],{"class":212,"line":688},[210,1672,1673],{"class":227},"        app.state.http ",[210,1675,292],{"class":223},[210,1677,1678],{"class":227}," client\n",[210,1680,1681],{"class":212,"line":703},[210,1682,1683],{"class":223},"        yield\n",[210,1685,1686],{"class":212,"line":709},[210,1687,257],{"emptyLinePlaceholder":256},[210,1689,1690],{"class":212,"line":717},[210,1691,257],{"emptyLinePlaceholder":256},[210,1693,1694,1696,1698,1701,1704,1706],{"class":212,"line":722},[210,1695,289],{"class":227},[210,1697,292],{"class":223},[210,1699,1700],{"class":227}," FastAPI(",[210,1702,1703],{"class":359},"lifespan",[210,1705,292],{"class":223},[210,1707,1708],{"class":227},"lifespan)\n",[210,1710,1711],{"class":212,"line":727},[210,1712,257],{"emptyLinePlaceholder":256},[210,1714,1715],{"class":212,"line":733},[210,1716,257],{"emptyLinePlaceholder":256},[210,1718,1719,1721,1723,1726,1729,1731],{"class":212,"line":749},[210,1720,312],{"class":311},[210,1722,315],{"class":227},[210,1724,1725],{"class":273},"\"\u002Fitems\u002F",[210,1727,1728],{"class":263},"{item_id}",[210,1730,1553],{"class":273},[210,1732,283],{"class":227},[210,1734,1735,1737,1739,1742,1745,1747],{"class":212,"line":773},[210,1736,326],{"class":223},[210,1738,329],{"class":223},[210,1740,1741],{"class":311}," get_item",[210,1743,1744],{"class":227},"(item_id: ",[210,1746,830],{"class":263},[210,1748,1749],{"class":227},", request: Request):\n",[210,1751,1752,1755,1757],{"class":212,"line":789},[210,1753,1754],{"class":227},"    client: httpx.AsyncClient ",[210,1756,292],{"class":223},[210,1758,1759],{"class":227}," request.app.state.http\n",[210,1761,1762,1764,1766,1768,1770,1772,1774,1776,1779,1782,1784],{"class":212,"line":794},[210,1763,347],{"class":227},[210,1765,292],{"class":223},[210,1767,780],{"class":223},[210,1769,696],{"class":227},[210,1771,1544],{"class":223},[210,1773,1725],{"class":273},[210,1775,1535],{"class":263},[210,1777,1778],{"class":227},"item_id",[210,1780,1781],{"class":263},"}",[210,1783,1553],{"class":273},[210,1785,283],{"class":227},[210,1787,1788],{"class":212,"line":801},[210,1789,625],{"class":227},[210,1791,1792,1794],{"class":212,"line":806},[210,1793,373],{"class":223},[210,1795,376],{"class":227},[14,1797,1798,1799,1802,1803,1805,1806,980],{},"One client per process, created inside the running loop, closed on shutdown. Because ",[18,1800,1801],{},"AsyncClient"," binds to the loop that created it, a module-level client built at import time will work in development and then fail with cryptic loop errors under a multi-worker launch — one more reason the lifespan is the right home. If you run multiple workers, remember the pool is per process: four workers with ",[18,1804,1393],{}," present 400 sockets to the upstream, which is worth checking against your supplier's limits before you scale the ",[26,1807,1809],{"href":1808},"\u002Fgetting-started-with-python-apis-for-builders\u002Fsetting-up-fastapi\u002Fuvicorn-vs-gunicorn-worker-configuration\u002F","worker count in Uvicorn or Gunicorn",[75,1811,84,1816,84,1819,84,1822,84,1825,84,1829,84,1834,84,1838,84,1842,84,1846,84,1849,84,1853,84,1855,84,1857,84,1859,84,1861,84,1865,84,1868,84,1870,84,1872,84,1874,84,1879,84,1883,84,1886,84,1889,84,1891,84,1893,84,1895,84,1898,84,1901,84,1904,84,1908,84,1911,84,1914,84,1916,84,1918],{"viewBox":1812,"role":78,"ariaLabelledBy":1813,"xmlns":82,"style":83},"0 0 720 330",[1814,1815],"hvr-life-t","hvr-life-d",[86,1817,1818],{"id":1814},"Handshake cost of a per-request client versus a shared client",[90,1820,1821],{"id":1815},"Three sequential requests each pay a ninety millisecond handshake with a per-request client, totalling 630 milliseconds, while a shared client pays it once for a total of 450 milliseconds.",[94,1823],{"x":96,"y":96,"width":97,"height":1824,"fill":99},"330",[101,1826,1828],{"x":404,"y":1827,"fill":105,"style":406},"26","New AsyncClient per request",[101,1830,1833],{"x":1831,"y":1832,"fill":138,"style":417},"80","63","req 1",[94,1835],{"x":1836,"y":1297,"width":1837,"height":405,"rx":1299,"fill":115,"stroke":116,"style":124},"90","108",[101,1839,1841],{"x":1840,"y":1832,"fill":105,"style":128},"144","TLS 90ms",[94,1843],{"x":1844,"y":1297,"width":1840,"height":405,"rx":1299,"fill":115,"stroke":1845,"style":124},"198","var(--c-blue)",[101,1847,1848],{"x":401,"y":1832,"fill":105,"style":128},"GET 120ms",[101,1850,1852],{"x":1831,"y":1851,"fill":138,"style":417},"99","req 2",[94,1854],{"x":1836,"y":1831,"width":1837,"height":405,"rx":1299,"fill":115,"stroke":116,"style":124},[101,1856,1841],{"x":1840,"y":1851,"fill":105,"style":128},[94,1858],{"x":1844,"y":1831,"width":1840,"height":405,"rx":1299,"fill":115,"stroke":1845,"style":124},[101,1860,1848],{"x":401,"y":1851,"fill":105,"style":128},[101,1862,1864],{"x":1831,"y":1863,"fill":138,"style":417},"135","req 3",[94,1866],{"x":1836,"y":1867,"width":1837,"height":405,"rx":1299,"fill":115,"stroke":116,"style":124},"116",[101,1869,1841],{"x":1840,"y":1863,"fill":105,"style":128},[94,1871],{"x":1844,"y":1867,"width":1840,"height":405,"rx":1299,"fill":115,"stroke":1845,"style":124},[101,1873,1848],{"x":401,"y":1863,"fill":105,"style":128},[101,1875,1878],{"x":1876,"y":1851,"fill":105,"style":1877},"366","font-size:13;font-family:var(--font-sans);","630 ms of wall clock",[101,1880,1882],{"x":404,"y":1881,"fill":105,"style":406},"186","One client for the process lifetime",[101,1884,1833],{"x":1831,"y":1885,"fill":138,"style":417},"223",[94,1887],{"x":1836,"y":1888,"width":1837,"height":405,"rx":1299,"fill":115,"stroke":116,"style":124},"204",[101,1890,1841],{"x":1840,"y":1885,"fill":105,"style":128},[94,1892],{"x":1844,"y":1888,"width":1840,"height":405,"rx":1299,"fill":115,"stroke":1845,"style":124},[101,1894,1848],{"x":401,"y":1885,"fill":105,"style":128},[101,1896,1852],{"x":1831,"y":1897,"fill":138,"style":417},"259",[94,1899],{"x":1836,"y":1900,"width":1840,"height":405,"rx":1299,"fill":115,"stroke":158,"style":124},"240",[101,1902,1848],{"x":1903,"y":1897,"fill":105,"style":128},"162",[101,1905,1907],{"x":1906,"y":1897,"fill":138,"style":428},"244","keep-alive reuse",[101,1909,1864],{"x":1831,"y":1910,"fill":138,"style":417},"295",[94,1912],{"x":1836,"y":1913,"width":1840,"height":405,"rx":1299,"fill":115,"stroke":158,"style":124},"276",[101,1915,1848],{"x":1903,"y":1910,"fill":105,"style":128},[101,1917,1907],{"x":1906,"y":1910,"fill":138,"style":428},[101,1919,1920],{"x":1876,"y":1897,"fill":105,"style":1877},"450 ms of wall clock",[42,1922],{},[45,1924,1926],{"id":1925},"failure-modes-async-introduces-that-requests-never-had","Failure modes async introduces that requests never had",[14,1928,1929],{},"Blocking code fails in boring ways. Async code fails under load, so budget an afternoon for these four.",[14,1931,1932,1935,1936,1938,1939,1941,1942,1945],{},[988,1933,1934],{},"Redirects are off by default."," ",[18,1937,20],{}," follows redirects on GET; ",[18,1940,58],{}," does not follow them anywhere unless you pass ",[18,1943,1944],{},"follow_redirects=True",". A port that \"works\" but suddenly returns empty 301 bodies is almost always this.",[14,1947,1948,1935,1951,1954,1955,1958,1959,1954,1962,1965,1966,1969],{},[988,1949,1950],{},"Exception names changed.",[18,1952,1953],{},"requests.exceptions.HTTPError"," becomes ",[18,1956,1957],{},"httpx.HTTPStatusError",", and ",[18,1960,1961],{},"requests.exceptions.RequestException",[18,1963,1964],{},"httpx.HTTPError",". Any ",[18,1967,1968],{},"except"," clause you miss turns a handled upstream failure into a 500 for a paying customer.",[14,1971,1972,1975,1976,1978,1979,1981,1982,1986],{},[988,1973,1974],{},"Pool exhaustion looks like an upstream outage."," When concurrency exceeds ",[18,1977,1268],{},", calls queue and then raise ",[18,1980,952],{},". The same shape shows up on the database side, and the diagnosis is identical — see ",[26,1983,1985],{"href":1984},"\u002Fscaling-and-operating-production-python-apis\u002Fasync-database-access-with-sqlalchemy\u002Ffixing-connection-pool-exhaustion\u002F","fixing connection pool exhaustion"," for the counting method.",[14,1988,1989,1992,1993,1997,1998,2002],{},[988,1990,1991],{},"Real concurrency finds your supplier's ceiling."," Fifty overlapping calls where you used to send one at a time is exactly how a quiet integration starts returning 429s; pair the migration with ",[26,1994,1996],{"href":1995},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fbest-practices-for-api-rate-limiting\u002F","rate limiting practices"," and read ",[26,1999,2001],{"href":2000},"\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fdebugging-429-too-many-requests-errors\u002F","debugging 429 errors"," before you raise the fan-out.",[14,2004,2005,2006,2008,2009,2012,2013,2017,2018,980],{},"One genuine bonus: ",[18,2007,58],{}," streams properly. ",[18,2010,2011],{},"client.stream(\"GET\", url)"," gives you an async iterator over chunks, which is how you handle ",[26,2014,2016],{"href":2015},"\u002Fgetting-started-with-python-apis-for-builders\u002Fparsing-json-responses\u002Fhandling-large-json-payloads-with-streaming\u002F","large JSON payloads"," without loading 200 MB into memory, and it is the same mechanism behind ",[26,2019,2021],{"href":2020},"\u002Fautomating-side-hustle-operations-with-apis\u002Fautomating-ai-workflows-with-python-apis\u002Fstreaming-llm-responses-through-fastapi\u002F","streaming LLM responses through FastAPI",[42,2023],{},[45,2025,2027],{"id":2026},"migrating-without-a-rewrite","Migrating without a rewrite",[14,2029,2030],{},"Do it in one pass per module, not as a big-bang branch. The surface is close enough that most edits are mechanical.",[2032,2033,2034,2048],"table",{},[2035,2036,2037],"thead",{},[2038,2039,2040,2043,2045],"tr",{},[2041,2042,20],"th",{},[2041,2044,58],{},[2041,2046,2047],{},"Watch for",[2049,2050,2051,2067,2081,2096],"tbody",{},[2038,2052,2053,2059,2064],{},[2054,2055,2056],"td",{},[18,2057,2058],{},"Session()",[2054,2060,2061],{},[18,2062,2063],{},"Client()",[2054,2065,2066],{},"Same idea, pooled",[2038,2068,2069,2074,2078],{},[2054,2070,2071],{},[18,2072,2073],{},"allow_redirects=True",[2054,2075,2076],{},[18,2077,1944],{},[2054,2079,2080],{},"Off by default",[2038,2082,2083,2088,2093],{},[2054,2084,2085],{},[18,2086,2087],{},"HTTPError",[2054,2089,2090],{},[18,2091,2092],{},"HTTPStatusError",[2054,2094,2095],{},"Rename every catch",[2038,2097,2098,2104,2109],{},[2054,2099,2100,2103],{},[18,2101,2102],{},"responses"," mock",[2054,2105,2106,2103],{},[18,2107,2108],{},"respx",[2054,2110,2111],{},"Rewrite fixtures",[14,2113,2114,2115,2119,2120,2124,2125,2129],{},"Start with the test suite, because that is where a silent behaviour change hides. Swap fixtures to ",[26,2116,2118],{"href":2117},"\u002Fscaling-and-operating-production-python-apis\u002Ftesting-python-apis-with-pytest\u002Fmocking-external-apis-with-respx\u002F","respx for mocking external APIs",", then exercise your own routes through ",[26,2121,2123],{"href":2122},"\u002Fscaling-and-operating-production-python-apis\u002Ftesting-python-apis-with-pytest\u002Ftesting-async-fastapi-endpoints-with-httpx\u002F","an async httpx test client"," so the transport under test is the transport you ship. Add a request hook that emits status, elapsed time and host into your ",[26,2126,2128],{"href":2127},"\u002Fscaling-and-operating-production-python-apis\u002Fmonitoring-and-logging-python-apis\u002Fstructured-logging-with-structlog\u002F","structured logs"," before the cutover, so you can compare p95 latency across the switch instead of guessing.",[14,2131,2132,2133,2137],{},"Commercially, the migration pays at peak, not on average. One million requests a month is roughly 0.4 per second, and at that rate nobody notices which client you chose. The bill arrives when a customer fires a 5,000-row batch and your peak jumps to 250 requests per second. Blocking at 8.3 rps per worker, that peak needs about 30 workers — fifteen 2 GB instances at roughly $25 each, $375 a month to absorb a burst lasting twenty seconds. The awaited version handles the same peak on one instance with headroom, so you keep two for redundancy and pay $50. That gap is margin, and it grows with every customer who integrates a batch job. Fold it into your ",[26,2134,2136],{"href":2135},"\u002Fbuilding-monetizing-api-driven-micro-saas\u002Fdesigning-api-pricing-tiers\u002Fcalculating-cost-per-api-request\u002F","cost per API request"," before you price a tier.",[42,2139],{},[45,2141,2143],{"id":2142},"builder-verdict","Builder verdict",[14,2145,2146,2147,2149,2150,2152,2153,2155,2156,2158,2159,2161],{},"Use ",[18,2148,58],{}," for anything async, full stop. The moment your handlers are ",[18,2151,189],{},", a blocking ",[18,2154,20],{}," call freezes the whole worker and erases the concurrency you paid for, and the fix is a one-line swap to an awaited client. For synchronous scripts and cron jobs, ",[18,2157,20],{}," remains dependable and there is no urgency to migrate working code. But if you are starting fresh on a commercial API that calls other APIs, standardise on ",[18,2160,58],{},": default timeouts, explicit per-phase control, first-class pooling, HTTP\u002F2 behind one flag, and one client that follows you from prototype to production without a second rewrite. Then spend the hour that matters — one client in the lifespan, four timeout phases sized to your latency promise, a justified pool limit, and a semaphore on every fan-out.",[45,2163,2165],{"id":2164},"faq","FAQ",[14,2167,2168,2171],{},[988,2169,2170],{},"Does switching to httpx actually lower my hosting bill?","\nOnly if your traffic is bursty, but then it lowers it a lot. A million requests a month averages under one per second and any client copes. The saving shows up at peak: a 250 rps burst against a 120 ms upstream needs roughly fifteen 2 GB instances when each worker blocks at 8.3 rps, versus one or two when calls overlap — about $375 a month against $50.",[14,2173,2174,2177],{},[988,2175,2176],{},"How risky is this migration on a live API?","\nLow, if you move one module at a time and start with tests. The two changes that bite are redirects being off by default and the renamed exception classes, both of which a decent test suite catches in minutes. Ship behind an env var that selects the client, run both for a day, then compare p95 latency and error rate before deleting the old path.",[14,2179,2180,2183,2184,2186],{},[988,2181,2182],{},"Can I keep requests anywhere in the codebase?","\nYes, and you should not force a rewrite. Scripts, cron-driven data pulls, migration one-offs and anything that runs in its own process are fine on ",[18,2185,20],{}," — blocking a process that has nothing else to do costs nothing. The rule is narrower than \"never use requests\": never call it inside an event loop.",[14,2188,2189,2192,2193,2195],{},[988,2190,2191],{},"Is HTTP\u002F2 worth the extra dependency?","\nUsually yes when you fan out many small calls to one host, because multiplexing removes the per-connection ceiling and shrinks your pool needs. It is not free — you add the ",[18,2194,45],{}," package and a little CPU per request — and it does nothing for a single large download. Enable it behind an env var, measure p95 on your real traffic, and keep it off if the number does not move.",[14,2197,2198,2201,2202,2205,2206,980],{},[988,2199,2200],{},"What breaks first when I raise concurrency?","\nYour supplier's rate limit, almost always before your own hardware. Real concurrency turns a polite serial integration into a burst, and the first symptom is 429s, sometimes followed by a temporary key suspension. Cap the fan-out with a semaphore, honour ",[18,2203,2204],{},"Retry-After",", and confirm your contracted limit before you raise ",[18,2207,1268],{},[45,2209,2211],{"id":2210},"related","Related",[14,2213,2214],{},[988,2215,2216],{},"Same topic area:",[2218,2219,2220,2226,2232],"ul",{},[2221,2222,2223,2225],"li",{},[26,2224,29],{"href":28}," — the parent guide covering sessions, headers and error handling.",[2221,2227,2228,2231],{},[26,2229,2230],{"href":997},"Retrying Failed HTTP Requests with tenacity"," — the retry policy that wraps the client you just chose.",[2221,2233,2234,2237],{},[26,2235,2236],{"href":2000},"Debugging 429 Too Many Requests Errors"," — read this before you raise concurrency on a live integration.",[14,2239,2240],{},[988,2241,2242],{},"Adjacent areas:",[2218,2244,2245,2251],{},[2221,2246,2247,2250],{},[26,2248,2249],{"href":2122},"Testing Async FastAPI Endpoints with httpx"," — exercise your routes through the same transport you ship.",[2221,2252,2253,2256],{},[26,2254,2255],{"href":1808},"Uvicorn vs Gunicorn Worker Configuration"," — worker count decides how many pools you actually run.",[2258,2259,2260],"style",{},"html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}html pre.shiki code .szBVR, html code.shiki .szBVR{--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .sVt8B, html code.shiki .sVt8B{--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .s4XuR, html code.shiki .s4XuR{--shiki-default:#E36209;--shiki-dark:#FFAB70}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":206,"searchDepth":220,"depth":220,"links":2262},[2263,2264,2265,2266,2267,2268,2269,2270,2271,2272],{"id":47,"depth":220,"text":48},{"id":182,"depth":220,"text":183},{"id":483,"depth":220,"text":484},{"id":958,"depth":220,"text":959},{"id":1379,"depth":220,"text":1380},{"id":1925,"depth":220,"text":1926},{"id":2026,"depth":220,"text":2027},{"id":2142,"depth":220,"text":2143},{"id":2164,"depth":220,"text":2165},{"id":2210,"depth":220,"text":2211},"httpx vs requests for async Python: why a blocking call freezes a FastAPI worker, per-phase timeouts, pool sizing, client lifetime, and the migration path.","md",{"pageTitle":2276,"type":2277,"datePublished":2278,"dateModified":2279},"httpx vs requests for Async Python API Calls in Production","article","2026-06-18","2026-07-23","\u002Fgetting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fhttpx-vs-requests-for-async",{"title":5,"description":2273},"getting-started-with-python-apis-for-builders\u002Fmaking-http-requests-with-requests-library\u002Fhttpx-vs-requests-for-async\u002Findex","qGToKFKeM-85xslJR8F-1A-OyZqiN7OLnSRbW6tC1aQ",{"@context":2285,"@type":2286,"mainEntity":2287},"https:\u002F\u002Fschema.org","FAQPage",[2288,2293,2296,2299,2302],{"@type":2289,"name":2170,"acceptedAnswer":2290},"Question",{"@type":2291,"text":2292},"Answer","Only if your traffic is bursty, but then it lowers it a lot. A million requests a month averages under one per second and any client copes. The saving shows up at peak: a 250 rps burst against a 120 ms upstream needs roughly fifteen 2 GB instances when each worker blocks at 8.3 rps, versus one or two when calls overlap — about $375 a month against $50.",{"@type":2289,"name":2176,"acceptedAnswer":2294},{"@type":2291,"text":2295},"Low, if you move one module at a time and start with tests. The two changes that bite are redirects being off by default and the renamed exception classes, both of which a decent test suite catches in minutes. Ship behind an env var that selects the client, run both for a day, then compare p95 latency and error rate before deleting the old path.",{"@type":2289,"name":2182,"acceptedAnswer":2297},{"@type":2291,"text":2298},"Yes, and you should not force a rewrite. Scripts, cron-driven data pulls, migration one-offs and anything that runs in its own process are fine on requests — blocking a process that has nothing else to do costs nothing. The rule is narrower than \"never use requests\": never call it inside an event loop.",{"@type":2289,"name":2191,"acceptedAnswer":2300},{"@type":2291,"text":2301},"Usually yes when you fan out many small calls to one host, because multiplexing removes the per-connection ceiling and shrinks your pool needs. It is not free — you add the h2 package and a little CPU per request — and it does nothing for a single large download. Enable it behind an env var, measure p95 on your real traffic, and keep it off if the number does not move.",{"@type":2289,"name":2200,"acceptedAnswer":2303},{"@type":2291,"text":2304},"Your supplier's rate limit, almost always before your own hardware. Real concurrency turns a polite serial integration into a burst, and the first symptom is 429s, sometimes followed by a temporary key suspension. Cap the fan-out with a semaphore, honour Retry-After, and confirm your contracted limit before you raise max_connections.",1784887028359]