Making HTTP Requests with the requests Library: A Production Client

Every commercial API you ship is also a client of somebody else's API. You call Stripe to charge a card, an LLM provider to generate text, a CRM to sync a contact. The requests library is where most Python builders make those calls, and it is where most of them quietly lose money — to connections that are never reused, to timeouts that were never set, to retries that hammer a provider until the account gets throttled. This guide rebuilds requests into a client you can put in front of paying customers. Part of the Getting Started with Python APIs for Builders guide.

The recommendation up front: use requests for synchronous work — cron jobs, background workers, management commands, data pipelines — and reach for httpx the moment the calling code lives inside an event loop. The reason is not style. A blocking requests call inside an async route stops the entire loop, so one slow provider stalls every concurrent user on that worker. The httpx vs requests for async comparison covers the migration path. Everything below — pooling, timeouts, retry policy, rate-limit handling — transfers to httpx almost line for line, so none of this work is wasted if you switch later.

Prerequisites

You need Python 3.11 or newer — the error-routing code below uses match statements and tomllib for config, both standard from 3.11. Install the client stack with pinned versions:

python -m pip install "requests==2.32.3" "urllib3==2.2.3" "responses==0.25.3"

Pin urllib3 explicitly, not just transitively. requests delegates pooling and retry behaviour to urllib3, and the retry semantics changed between urllib3 1.x and 2.x: method_whitelist became allowed_methods, and Retry gained stricter defaults around Retry-After. A machine that resolves a different urllib3 than your laptop will retry differently in production, and that class of drift produces the worst kind of bug — one that only appears under provider failure.

Set these environment variables before the client starts. Never hardcode the base URL: staging and production almost always differ, and a hardcoded host is how test traffic ends up billed against your live account. Key handling belongs to API authentication, and if you support more than one key at a time, read rotating API keys without downtime before you build the refresh path.

export PROVIDER_API_BASE="https://api.example.com/v1"
export PROVIDER_API_KEY="sk_live_..."
export HTTP_CONNECT_TIMEOUT="3.05"
export HTTP_READ_TIMEOUT="10"
export HTTP_POOL_MAXSIZE="32"
export HTTP_RETRY_TOTAL="3"
export HTTP_BACKOFF_FACTOR="0.5"

If you are still finding your footing with the basics of the library, how to use Python requests for beginners covers the fundamentals this guide assumes.

Step 1: Build one Session per process, not one per call

requests.get() is a convenience wrapper that creates a Session, makes a single call, and throws the session away. Every call therefore pays for a DNS lookup, a TCP three-way handshake and a full TLS negotiation. Against a typical cloud API that is 130–170 ms of pure setup before the provider even reads your request. Do it 200 times in a loop and you have burned close to 30 seconds on handshakes you did not need.

A Session holds a urllib3 connection pool. The first call to a host performs the handshake; every subsequent call to the same host borrows the already-open, already-negotiated socket. The measurements below come from 200 sequential GETs to a JSON endpoint over TLS from a small cloud VM in the same region — the shape of the result holds anywhere, only the absolute numbers move.

# api_client/session.py
import os
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

API_BASE = os.getenv("PROVIDER_API_BASE", "http://localhost:8000")


def build_session() -> requests.Session:
    """One session per process. Thread-safe for sharing across worker threads."""
    api_key = os.getenv("PROVIDER_API_KEY")
    if not api_key:
        raise RuntimeError("PROVIDER_API_KEY is not set")

    retry = Retry(
        total=int(os.getenv("HTTP_RETRY_TOTAL", "3")),
        backoff_factor=float(os.getenv("HTTP_BACKOFF_FACTOR", "0.5")),
        status_forcelist=(429, 500, 502, 503, 504),
        allowed_methods=frozenset({"GET", "HEAD", "PUT", "DELETE", "OPTIONS"}),
        respect_retry_after_header=True,
        raise_on_status=False,
    )
    adapter = HTTPAdapter(
        max_retries=retry,
        pool_connections=int(os.getenv("HTTP_POOL_CONNECTIONS", "16")),
        pool_maxsize=int(os.getenv("HTTP_POOL_MAXSIZE", "32")),
        pool_block=True,
    )

    session = requests.Session()
    session.mount("https://", adapter)
    session.mount("http://", adapter)
    session.headers.update(
        {
            "Accept": "application/json",
            "User-Agent": os.getenv("HTTP_USER_AGENT", "builder-client/1.0"),
            "Authorization": f"Bearer {api_key}",
        }
    )
    return session

Three arguments there earn their keep. pool_connections is the number of distinct hosts the adapter keeps pools for — 16 is generous for a client that talks to two or three providers. pool_maxsize is the number of sockets held open per host, and it must be at least as large as the number of threads that will use the session concurrently. pool_block=True is the one people skip and regret: without it, a thread that finds the pool empty silently opens a throwaway connection, so under load you get the handshake cost back plus a warning-log flood about discarded connections. With it, the thread waits for a free socket, which is what you actually want.

Share one session per process. A Session is safe to use from multiple threads as long as you do not mutate session.headers while calls are in flight — set headers once at construction, and pass per-call overrides through the headers= argument.

The third bar is the one that matters commercially. Pooling alone cuts per-call latency by roughly 78 percent, but throughput only jumps when you also raise pool_maxsize and run several threads: 5.4 calls per second becomes about 180. For a batch job that syncs 50,000 records nightly, that is the difference between a nine-hour run and a five-minute one — and on a per-second billed platform, the compute bill follows the wall clock.

Step 2: Set two timeouts on every call, and make them the default

The single most expensive default in requests is timeout=None. With no timeout, a call waits forever. Not for 30 seconds, not until the platform kills it — forever. One provider that accepts a connection and then never responds will pin a worker thread until the process restarts. Do that across a handful of workers and your paid API returns 503 to customers while every dashboard shows the CPU at three percent.

Pass a two-tuple: (connect_timeout, read_timeout). The connect timeout bounds DNS, TCP and TLS; use 3.05 seconds, because the extra 0.05 keeps the value just above the multiples-of-three TCP retransmission window and avoids a class of spurious failures. The read timeout bounds the gap between bytes, not the total duration of the response — a streaming endpoint that sends a chunk every 8 seconds will never trip a 10-second read timeout, no matter how long the whole download takes.

Setting the timeout at every call site works until someone forgets. Push the default down into the adapter so a forgotten argument cannot produce an unbounded call:

# api_client/timeout.py
import os
import requests
from requests.adapters import HTTPAdapter

DEFAULT_TIMEOUT = (
    float(os.getenv("HTTP_CONNECT_TIMEOUT", "3.05")),
    float(os.getenv("HTTP_READ_TIMEOUT", "10")),
)


class TimeoutHTTPAdapter(HTTPAdapter):
    """Applies a default timeout to any request that does not set one."""

    def __init__(self, *args, timeout: tuple[float, float] = DEFAULT_TIMEOUT, **kwargs):
        self._timeout = timeout
        super().__init__(*args, **kwargs)

    def send(self, request, **kwargs):
        if kwargs.get("timeout") is None:
            kwargs["timeout"] = self._timeout
        return super().send(request, **kwargs)


def with_default_timeout(session: requests.Session) -> requests.Session:
    for prefix, adapter in list(session.adapters.items()):
        session.mount(
            prefix,
            TimeoutHTTPAdapter(
                max_retries=adapter.max_retries,
                pool_connections=adapter._pool_connections,
                pool_maxsize=adapter._pool_maxsize,
                pool_block=adapter._pool_block,
            ),
        )
    return session

Now do the arithmetic that most teams skip. Worst-case wall time for one logical call is (connect + read) × attempts + total backoff. With the settings above — 3.05 + 10 seconds, four attempts, backoff sleeps of 0, 1.0 and 2.0 seconds — a fully unresponsive provider holds a worker for 55.2 seconds. If your own API promises a p99 under two seconds, that number is unacceptable, and the fix is not a shorter timeout: it is moving the call out of the request path into a background job or serving a cached answer from Redis.

Step 3: Retry only what is safe to retry

Retries are insurance, and like insurance they are cheap until you buy too much. The Retry object above retries on 429, 500, 502, 503 and 504, and only for methods listed in allowed_methods. That list deliberately excludes POST.

The reason is idempotency. A GET repeated three times reads the same resource three times — harmless. A POST repeated three times can create three charges, three subscriptions, three emails. Worse, the failure mode that triggers a retry is usually a read timeout, which means the provider may well have processed your first request and simply failed to tell you. Retrying blind turns a network hiccup into a duplicate charge and a support ticket.

If the provider supports idempotency keys — Stripe does, and so do most serious billing APIs — you can safely retry a POST by sending the same key each time. Generate it once, outside the retry loop, and store it with the operation:

# api_client/idempotent.py
import os
import uuid
import requests

API_BASE = os.getenv("PROVIDER_API_BASE", "http://localhost:8000")


def create_charge(session: requests.Session, payload: dict, key: str | None = None) -> dict:
    """POST with an idempotency key so a retry cannot double-charge."""
    idempotency_key = key or str(uuid.uuid4())
    response = session.post(
        f"{API_BASE}/charges",
        json=payload,
        headers={"Idempotency-Key": idempotency_key},
    )
    response.raise_for_status()
    return response.json()

Use json=payload rather than data=json.dumps(payload). It serialises the body and sets Content-Type: application/json for you, and it removes a whole family of encoding bugs where a manually built body disagrees with the declared content type.

On backoff maths: urllib3 computes the sleep as backoff_factor × 2 ** (attempt - 1), and it does not sleep before the first retry. With backoff_factor=0.5 you get 0 s, 1 s, 2 s. With backoff_factor=2 you get 0 s, 4 s, 8 s — safer for the provider, brutal for your latency budget. Start at 0.5 and raise it only if the provider's support team asks you to. When you need retry logic richer than urllib3 offers — per-exception policies, custom stop conditions, callbacks that emit metrics — move up to retrying failed HTTP requests with tenacity, which wraps your own function instead of the transport layer.

Step 4: Route every failure into a named outcome

requests raises a small family of exceptions and returns everything else as a Response. Production code has to collapse both into a decision: retry, fail loudly, or degrade. A match statement makes that routing readable and keeps the policy in one place instead of scattered across call sites.

# api_client/call.py
import os
import requests

API_BASE = os.getenv("PROVIDER_API_BASE", "http://localhost:8000")


class ProviderError(Exception):
    """Base class for every provider failure."""


class ProviderRetryable(ProviderError):
    """Transient: safe to try again later."""


class ProviderFatal(ProviderError):
    """Permanent: the call itself is wrong. Alert, do not retry."""


def call(session: requests.Session, method: str, path: str, **kwargs) -> dict:
    url = f"{API_BASE}{path}"
    try:
        response = session.request(method, url, **kwargs)
    except requests.exceptions.ConnectTimeout as exc:
        raise ProviderRetryable("connect timeout") from exc
    except requests.exceptions.ReadTimeout as exc:
        raise ProviderRetryable("read timeout") from exc
    except requests.exceptions.SSLError as exc:
        raise ProviderFatal("TLS verification failed") from exc
    except requests.exceptions.ConnectionError as exc:
        raise ProviderRetryable("connection reset or DNS failure") from exc

    match response.status_code:
        case code if 200 <= code < 300:
            try:
                return response.json()
            except requests.exceptions.JSONDecodeError as exc:
                raise ProviderFatal(f"non-JSON body: {response.text[:200]}") from exc
        case 401 | 403:
            raise ProviderFatal(f"auth rejected: {response.text[:200]}")
        case 404:
            raise ProviderFatal(f"no such resource: {path}")
        case 409:
            raise ProviderFatal("conflict: the resource already exists")
        case 422:
            raise ProviderFatal(f"payload rejected: {response.text[:200]}")
        case 429:
            raise ProviderRetryable(f"rate limited, retry-after={response.headers.get('Retry-After')}")
        case code if code >= 500:
            raise ProviderRetryable(f"provider {code}")
        case code:
            raise ProviderFatal(f"unhandled status {code}")

Two details are load-bearing. First, SSLError is fatal, not retryable — a certificate problem never fixes itself on attempt two, and retrying just delays the alert. Second, truncate response.text before it reaches your logs. A provider that returns an HTML error page will otherwise dump kilobytes per failure into your log pipeline, and log ingestion is billed by volume; a 200-character slice keeps the diagnostic value at a fraction of the cost. Pair this with structured logging so each failure carries the status code, host and elapsed time as queryable fields rather than prose.

Notice that a 2xx response can still be a failure. Some APIs return 200 OK with an error object in the body, and some return HTML from a load balancer that never reached the application. Never trust the status code alone — validate the parsed body against a schema. Validating JSON with Pydantic v2 covers the model-based approach, and the wider parsing JSON responses guide handles the messier shapes.

Step 5: Stay inside the provider's rate limit

Getting a 429 is a design failure, not an accident. Most providers publish your remaining budget on every response, so a client that reads those headers can slow itself down before it gets thrown out. The headers are not standardised — X-RateLimit-Remaining and X-RateLimit-Reset are the common pair, RateLimit-Remaining and RateLimit-Reset follow the newer draft, and Retry-After may be either a number of seconds or an HTTP date. Handle both forms:

# api_client/limits.py
import os
import time
from email.utils import parsedate_to_datetime
from datetime import datetime, timezone

import requests

FLOOR = int(os.getenv("HTTP_RATE_LIMIT_FLOOR", "5"))


def seconds_until_reset(response: requests.Response) -> float:
    """Read Retry-After or a reset header and return a sleep duration."""
    retry_after = response.headers.get("Retry-After")
    if retry_after:
        if retry_after.isdigit():
            return float(retry_after)
        try:
            when = parsedate_to_datetime(retry_after)
            return max(0.0, (when - datetime.now(timezone.utc)).total_seconds())
        except (TypeError, ValueError):
            return 1.0

    for header in ("X-RateLimit-Reset", "RateLimit-Reset"):
        raw = response.headers.get(header)
        if raw and raw.isdigit():
            value = float(raw)
            # Large values are epoch seconds; small ones are a delta.
            return max(0.0, value - time.time()) if value > 1_000_000 else value
    return 1.0


def throttle(response: requests.Response) -> None:
    """Pause when the remaining budget drops below the floor."""
    for header in ("X-RateLimit-Remaining", "RateLimit-Remaining"):
        raw = response.headers.get(header)
        if raw is None or not raw.isdigit():
            continue
        if int(raw) <= FLOOR:
            time.sleep(seconds_until_reset(response))
        return

That epoch-versus-delta heuristic is not elegant, but it is the pragmatic answer to a real inconsistency: GitHub sends an absolute epoch timestamp, while many smaller APIs send a number of seconds. Guessing wrong in the other direction — treating an epoch as a delta — sleeps your worker for fifty-six years, so bias the check toward the safe interpretation and log which branch fired.

Keep the floor above zero. Sleeping only at zero remaining means every concurrent worker discovers the limit at the same instant and they all wake together, which produces exactly the burst that gets you throttled again. A floor of 5 with a small random jitter spreads the wake-ups. The dedicated best practices for API rate limiting guide covers token buckets shared across processes, and debugging 429 too many requests errors walks through diagnosing a limit you cannot see. If you are polling a provider on a schedule purely to notice changes, check whether they offer callbacks first — webhooks instead of polling usually cuts the call volume by an order of magnitude and removes the rate-limit problem entirely.

Step 6: Stream large responses instead of buffering them

response.json() loads the whole body into memory first. For a 400 MB export that is 400 MB of resident memory on a container you are probably paying 512 MB for, and the OOM killer takes the process down mid-job. Pass stream=True and consume in chunks:

# api_client/download.py
import os
import requests

MAX_BYTES = int(os.getenv("HTTP_MAX_DOWNLOAD_BYTES", str(200 * 1024 * 1024)))
CHUNK = int(os.getenv("HTTP_CHUNK_BYTES", "65536"))


def download(session: requests.Session, url: str, dest: str) -> int:
    written = 0
    with session.get(url, stream=True) as response:
        response.raise_for_status()
        declared = int(response.headers.get("Content-Length", "0"))
        if declared > MAX_BYTES:
            raise ValueError(f"refusing {declared} bytes, limit is {MAX_BYTES}")
        with open(dest, "wb") as handle:
            for chunk in response.iter_content(chunk_size=CHUNK):
                written += len(chunk)
                if written > MAX_BYTES:
                    raise ValueError("response exceeded the size limit mid-stream")
                handle.write(chunk)
    return written

The with block matters: without it, a streamed response holds its connection out of the pool until garbage collection, and a loop over a few hundred downloads will exhaust the pool. Check Content-Length and count bytes as they arrive, because a chunked response has no declared length and a hostile or broken provider can send far more than it promised. For payloads you must parse rather than store, handling large JSON payloads with streaming shows how to consume records incrementally.

Step 7: Test the client without touching the provider

Never point tests at the live API. You will pay for the calls, you will hit rate limits in CI, and you cannot reproduce a 503 on demand. The responses library patches the transport layer, so your real client code — session, adapter, retry config and all — runs unchanged against scripted replies.

# tests/test_client.py
import os
import pytest
import responses

from api_client.session import build_session
from api_client.call import call, ProviderRetryable, ProviderFatal

BASE = os.getenv("PROVIDER_API_BASE", "http://localhost:8000")


@pytest.fixture()
def session():
    os.environ.setdefault("PROVIDER_API_KEY", "test-key")
    client = build_session()
    yield client
    client.close()


@responses.activate
def test_retries_then_succeeds(session):
    responses.add(responses.GET, f"{BASE}/items", status=503)
    responses.add(responses.GET, f"{BASE}/items", json={"items": []}, status=200)

    assert call(session, "GET", "/items") == {"items": []}
    assert len(responses.calls) == 2


@responses.activate
def test_client_error_is_fatal(session):
    responses.add(responses.GET, f"{BASE}/items", json={"error": "bad key"}, status=401)

    with pytest.raises(ProviderFatal):
        call(session, "GET", "/items")


@responses.activate
def test_rate_limit_surfaces_as_retryable(session):
    responses.add(
        responses.GET,
        f"{BASE}/items",
        json={"error": "slow down"},
        status=429,
        headers={"Retry-After": "1"},
    )

    with pytest.raises(ProviderRetryable):
        call(session, "GET", "/items")

Assert on len(responses.calls), not just the return value. That assertion is what catches a retry policy that silently stopped working after a urllib3 upgrade — the happy path still passes, but the call count drops from two to one. Wire these into the wider suite described in testing Python APIs with pytest; if you have already migrated to httpx, the equivalent tool is respx. Once the client is trustworthy, wire it into your own service by following setting up FastAPI — remembering to run blocking calls in a thread pool if that service is async.

Configuration reference

Every knob below is read from the environment by the code in this guide. The production column assumes a container with 1 vCPU serving a synchronous worker pool.

Size HTTP_POOL_MAXSIZE to your worker thread count plus a small headroom — four extra sockets absorb the moment when a slow call has not yet returned its connection. Lower HTTP_READ_TIMEOUT to 5 seconds for interactive paths where a customer is waiting, and raise it to 30 for report generation endpoints that legitimately take time. Keep HTTP_RETRY_TOTAL at 2 or 3: the marginal recovery from a fourth attempt is under one percent for most providers, while the worst-case latency grows linearly.

Gotchas and failure modes

• Creating a session per request. A Session built inside a request handler pools nothing. Build it once at import or in the application lifespan and reuse it.
• Mutating shared session headers per call. Setting session.headers["Authorization"] for a specific tenant leaks that key to whichever concurrent thread calls next. Pass per-call headers= instead.
• Relying on the default pool size under threads. The default pool_maxsize is 10. Run 32 threads against it without pool_block=True and you silently open and discard connections all day.
• Retrying POST without an idempotency key. The retry fires on a read timeout — precisely when the provider may have already succeeded. Duplicate charges follow.
• Ignoring Retry-After on 429. Retrying immediately extends the penalty window on most providers and gets keys suspended on a few.
• Leaving verify=False in the code after a local debugging session. It disables certificate validation everywhere the session is used and turns any network position into a credential theft opportunity.

Verification

Confirm the pool is actually being reused rather than trusting the config. Enable urllib3's debug logging and count how many times it says "Starting new HTTPS connection":

# scripts/verify_pool.py
import logging
import os
import time

from api_client.session import build_session

logging.basicConfig(level=logging.DEBUG)
path = os.getenv("PROVIDER_HEALTH_PATH", "/health")
base = os.getenv("PROVIDER_API_BASE", "http://localhost:8000")

session = build_session()
start = time.perf_counter()
for _ in range(10):
    session.get(f"{base}{path}")
elapsed = time.perf_counter() - start
session.close()

print(f"10 calls in {elapsed:.2f}s ({elapsed / 10 * 1000:.0f} ms per call)")

You want exactly one "Starting new HTTPS connection" line for ten calls. If you see ten, the session is not being reused — usually because the code path builds a fresh session, or because the host redirects to a different domain and the pool for that second host starts cold.

Cost and performance at scale

Work the numbers for a service making one million outbound calls a month. Without pooling, at 184 ms per call, that is 51 hours of wall-clock time held by your workers. With pooling at 41 ms, it is 11.4 hours. On a platform billing roughly $0.000021 per vCPU-second for a small always-on container, the difference is small in absolute dollars — a few dollars either way — but the concurrency difference is what actually costs money: 51 hours of blocked worker time on a 1 vCPU box needs three or four instances to keep up, while 11.4 hours fits comfortably in one. That is the difference between a $7/month plan and a $28/month one, before you account for the extra instances your own latency SLA would force you to buy.

Retries change the arithmetic in the other direction. If a provider fails 0.5 percent of calls and you retry up to three times, you add roughly 5,000 extra requests a month. Against a metered provider charging $0.002 per call, that is $10 of pure overhead — trivial. Set HTTP_RETRY_TOTAL=8 on a provider having a bad day, though, and the same failure rate can multiply your billed call volume by a noticeable factor while your own p99 collapses. Cap retries, and put the number in a dashboard next to your cost per API request so a change in provider reliability shows up in margin before it shows up in the invoice.

The largest saving is not in the client at all. Every call you avoid making costs nothing and takes zero milliseconds. Cache aggressively for read-heavy endpoints, batch where the provider supports it, and honour ETag and If-None-Match — a 304 response typically bills as a request but transfers no body, so it saves bandwidth and parsing even when it does not save quota.

FAQ

What does a pooled session actually save at one million outbound calls a month? Around 143 ms per call of handshake time, which at a million calls is roughly 40 hours of blocked worker time. In practice that removes two or three container instances from your bill — call it $20 to $60 a month on a small platform — and it cuts your own p95 latency enough that you may not need a bigger plan at all.

Should I migrate an existing requests codebase to httpx? Only if the calling code is async. A synchronous worker, cron job or data pipeline gains nothing from the migration and inherits a less battle-tested library. If you are inside an event loop, migrate immediately, because blocking calls there cap your concurrency at one. The httpx vs requests for async breakdown covers the porting cost, which is usually an afternoon.

How do I rotate a provider key without dropping in-flight requests? Build a new Session with the new key and swap the module-level reference atomically; existing calls finish on the old session, and the old session gets closed once its calls drain. Never mutate session.headers on a live session under threads. The full dual-key procedure is in rotating API keys without downtime.

At what point do retries cost more than the failures they fix? Past three attempts. Recovery rates on the fourth attempt are typically under one percent, while every extra attempt adds a full timeout window plus backoff to your worst-case latency and another billed call to the provider's meter. Cap at three and route persistent failures into a queue for later rather than retrying in the request path.

Can one slow provider take down my paid API? Yes, and it is the most common outage cause for small API businesses. With no timeout, a hung provider pins workers until the process restarts; with the settings in this guide, the worst case is still 55 seconds per call. Move third-party calls out of the request path into background jobs, and add a circuit breaker that stops calling a provider after a run of failures.

Related

• httpx vs requests for async — the migration decision once your calls live inside an event loop.
• Retrying failed HTTP requests with tenacity — when urllib3's retry policy is too blunt for your failure modes.
• Debugging 429 too many requests errors — diagnosing a limit the provider will not document.
• Parsing JSON responses — turning the bytes this client returns into validated objects.
• Caching Python API responses with Redis — the cheapest outbound call is the one you never make.