When to Use Webhooks Instead of Polling

You have a third-party integration that needs to react to something changing somewhere else: an order was paid, a document finished processing, a lead replied. You can ask repeatedly, or you can be told. That single decision sets your compute bill, your p50 freshness, and how often you get rate limited for the next two years. Part of the Understanding REST vs GraphQL guide, because the protocol conversation is incomplete until you decide who initiates the call.

The short version: webhooks win on almost every commercial axis the moment your event rate per watched resource drops below your freshness requirement, which is nearly always. But a webhook-only design is fragile, and a polling-only design is expensive. The answer you ship is a hybrid, and this page gives you the arithmetic and the code for it.

The Cost Crossover Is Arithmetic, Not Opinion

Polling cost scales with the number of watched resources multiplied by the polling frequency. Webhook cost scales with the number of actual events. Those are completely different curves, and they cross at a point you can calculate before writing a line of code.

Take a realistic side-hustle scale: 500 connected customer accounts, each of which you check for new activity. Poll every 60 seconds and you issue 500 × 60 × 24 × 30 = 21.6 million outbound requests a month. At roughly 120 ms of process time per request — connection reuse, TLS resumption, JSON decode, a database comparison — that is 720 CPU-hours. A calendar month is 730 hours. You are burning one saturated vCPU continuously, forever, to ask a question whose answer is "nothing changed" more than 99% of the time.

Now count the webhook side. If each of those 500 accounts genuinely produces 40 meaningful events a month, that is 20,000 inbound requests. Same 120 ms of work each, and the whole month costs 40 CPU-minutes. The difference is not a percentage. It is three orders of magnitude.

The crossover rule falls out of that comparison in one line: polling is cheaper only when each watched resource changes more often than once per polling interval. If a customer's Stripe account fires an event every ten seconds and you need one-minute freshness, polling batches six events into one request and wins. That situation is rare. For almost every integration a builder ships — payments, CRM records, file processing, inbox sync — resources sit idle for hours and change in bursts.

Encode the model rather than guessing. This script gives you a defensible number for a pricing conversation, and it pairs with the deeper treatment in calculating cost per API request.

import os
from dataclasses import dataclass

SECONDS_PER_MONTH = 30 * 24 * 3600


@dataclass(frozen=True)
class Workload:
    accounts: int
    poll_interval_s: int
    events_per_account_month: int
    cpu_seconds_per_request: float
    cpu_hour_cost: float

    @property
    def poll_requests(self) -> int:
        return self.accounts * (SECONDS_PER_MONTH // self.poll_interval_s)

    @property
    def webhook_requests(self) -> int:
        return self.accounts * self.events_per_account_month

    def cost(self, requests: int) -> float:
        return requests * self.cpu_seconds_per_request / 3600 * self.cpu_hour_cost


def load_workload() -> Workload:
    return Workload(
        accounts=int(os.getenv("WATCHED_ACCOUNTS", "500")),
        poll_interval_s=int(os.getenv("POLL_INTERVAL_SECONDS", "60")),
        events_per_account_month=int(os.getenv("EVENTS_PER_ACCOUNT", "40")),
        cpu_seconds_per_request=float(os.getenv("CPU_SECONDS_PER_REQUEST", "0.12")),
        cpu_hour_cost=float(os.getenv("CPU_HOUR_COST", "0.034")),
    )


if __name__ == "__main__":
    w = load_workload()
    poll_cost = w.cost(w.poll_requests)
    hook_cost = w.cost(w.webhook_requests)
    ratio = poll_cost / max(hook_cost, 1e-9)

    match ratio:
        case r if r > 20:
            verdict = "webhooks, decisively"
        case r if r > 2:
            verdict = "webhooks, with a reconciliation sweep"
        case _:
            verdict = "polling is competitive, keep it simple"

    print(f"poll:    {w.poll_requests:>12,} req  ${poll_cost:,.2f}")
    print(f"webhook: {w.webhook_requests:>12,} req  ${hook_cost:,.2f}")
    print(f"ratio {ratio:,.1f}x -> {verdict}")

Latency Is the Axis That Actually Sells

Cost convinces you. Latency convinces your customers. Polling produces a sawtooth staleness curve: immediately after a poll your data is fresh, and just before the next one it is a full interval old. Mean detection latency is half the interval, worst case is the whole interval plus the provider's own write propagation delay. A five-minute poll means the average customer waits 150 seconds to see their payment confirmed, and the unlucky ones wait 300.

A webhook flattens that curve. Providers typically dispatch within one to three seconds of the state change, so your p50 detection latency drops to single-digit seconds and your p99 is governed by their retry queue rather than your cron schedule. That is the difference between a dashboard that feels live and one that feels broken.

There is a second, meaner latency effect. Most providers cap you at something like 100 requests per minute per token. Five hundred accounts polled every minute needs 500 requests per minute, so you either shard across tokens or you stagger — and staggering silently stretches your real freshness from one minute to five while your monitoring still says "polling every 60 s". If you have ever chased that discrepancy, the mechanics are laid out in debugging 429 Too Many Requests errors and the budgeting side in best practices for API rate limiting.

Ship the Receiver as a Thin, Fast Acknowledgement

A webhook endpoint is a public, unauthenticated-by-default door into your system, and the provider will disable it if you are slow. Every serious provider expects a 2xx within a few seconds and retries with exponential backoff otherwise. So the receiver does exactly three things: verify the signature, persist the raw event, return 202. All real work happens elsewhere.

Read the body as raw bytes before anything else. Signature schemes hash the exact transmitted payload, and if you let a framework parse and re-serialise the JSON first, key ordering and whitespace changes will break verification in ways that look like a secret mismatch. This is a natural fit for an async FastAPI setup.

import hashlib
import hmac
import os

from fastapi import APIRouter, Header, HTTPException, Request, Response

router = APIRouter()
WEBHOOK_SECRET = os.environ["WEBHOOK_SIGNING_SECRET"].encode()
MAX_BODY_BYTES = int(os.getenv("WEBHOOK_MAX_BODY_BYTES", "1048576"))


def valid_signature(payload: bytes, provided: str) -> bool:
    expected = hmac.new(WEBHOOK_SECRET, payload, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, provided)


@router.post("/webhooks/provider", status_code=202)
async def receive(request: Request, x_signature: str = Header(default="")) -> Response:
    payload = await request.body()
    if len(payload) > MAX_BODY_BYTES:
        raise HTTPException(status_code=413, detail="payload too large")
    if not valid_signature(payload, x_signature):
        raise HTTPException(status_code=401, detail="bad signature")

    event = await request.json()
    await store_raw_event(
        event_id=event["id"],
        event_type=event["type"],
        payload=payload,
    )
    return Response(status_code=202)

store_raw_event writes to a table with a unique constraint on the provider's event id and swallows the conflict. Providers deliver at least once, not exactly once, so duplicates are normal traffic rather than an error condition — the full pattern lives in building an idempotent webhook receiver, and the provider-specific details of signature checking are covered in verifying Stripe webhook signatures. Draining the stored events into your domain model belongs in a worker; running background jobs with Celery covers the queue side.

Hybrid Reconciliation Is the Design That Survives

Webhooks fail silently. Your endpoint returns a 500 during a deploy, the provider retries for six hours and gives up, and nobody notices until a customer emails about a missing order three weeks later. A webhook-only architecture has no mechanism for discovering what it never received.

So run both. Webhooks carry the latency requirement; a slow, cheap sweep carries the correctness requirement. Once an hour — or nightly, depending on how much a missed record costs you — pull everything the provider changed since your last watermark and reconcile it against your local state. This sweep runs at a fraction of a percent of full polling cost because it fires once per account per interval, not once per account per minute.

The sweep is short. Page through changed records since your stored watermark, upsert each one through the same handler the webhook path uses, then advance the watermark only after the page commits. Use httpx for async work so a single process can reconcile many accounts concurrently, and schedule it through the patterns in scheduling data pipelines with cron.

import asyncio
import os
from datetime import datetime, timedelta, timezone

import httpx

API_BASE = os.environ["PROVIDER_API_BASE"]
API_TOKEN = os.environ["PROVIDER_API_TOKEN"]
PAGE_SIZE = int(os.getenv("SWEEP_PAGE_SIZE", "200"))
OVERLAP = timedelta(minutes=int(os.getenv("SWEEP_OVERLAP_MINUTES", "10")))


async def sweep(client: httpx.AsyncClient, account_id: str) -> int:
    watermark = await load_watermark(account_id)
    cursor: str | None = None
    seen = 0

    while True:
        params = {
            "account": account_id,
            "updated_since": (watermark - OVERLAP).isoformat(),
            "limit": PAGE_SIZE,
        }
        if cursor:
            params["cursor"] = cursor

        response = await client.get(f"{API_BASE}/events", params=params)
        response.raise_for_status()
        body = response.json()

        for record in body["data"]:
            await apply_event(record)
            seen += 1

        cursor = body.get("next_cursor")
        if not cursor:
            break

    await save_watermark(account_id, datetime.now(timezone.utc))
    return seen


async def main() -> None:
    headers = {"Authorization": f"Bearer {API_TOKEN}"}
    limits = httpx.Limits(max_connections=int(os.getenv("SWEEP_CONCURRENCY", "20")))
    async with httpx.AsyncClient(headers=headers, timeout=30.0, limits=limits) as client:
        accounts = await list_active_accounts()
        results = await asyncio.gather(*(sweep(client, a) for a in accounts))
    print(f"reconciled {sum(results)} records across {len(accounts)} accounts")


if __name__ == "__main__":
    asyncio.run(main())

Two details make this reliable. The overlap window re-reads a few minutes you already processed, which costs nothing because apply_event is idempotent and protects you from clock skew between your box and theirs. And advancing the watermark only after a successful page means a crash mid-sweep replays rather than skips.

When the Third Party Offers No Webhook

Plenty of APIs never send you anything. You still have better options than a naive fixed-interval loop, and they can cut effective request cost by 80% or more.

Conditional requests are the biggest single win. Send If-None-Match with the last ETag you saw; an unchanged resource returns a 304 with an empty body. You still pay a round trip and a rate-limit slot, but you skip the payload transfer and all deserialisation, which is where most of your CPU time goes. Layer adaptive intervals on top: back off when nothing changes, tighten up immediately after a change, because activity arrives in bursts.

import asyncio
import hashlib
import os

import httpx

RESOURCE_URL = os.environ["WATCHED_RESOURCE_URL"]
MIN_INTERVAL = float(os.getenv("POLL_MIN_SECONDS", "15"))
MAX_INTERVAL = float(os.getenv("POLL_MAX_SECONDS", "900"))
BACKOFF_FACTOR = float(os.getenv("POLL_BACKOFF_FACTOR", "1.6"))


async def watch(client: httpx.AsyncClient) -> None:
    interval = MIN_INTERVAL
    etag: str | None = None
    digest: str | None = None

    while True:
        headers = {"If-None-Match": etag} if etag else {}
        response = await client.get(RESOURCE_URL, headers=headers)

        match response.status_code:
            case 304:
                changed = False
            case 200:
                etag = response.headers.get("etag")
                new_digest = hashlib.sha256(response.content).hexdigest()
                changed = new_digest != digest
                digest = new_digest
                if changed:
                    await apply_snapshot(response.json())
            case 429:
                retry_after = float(response.headers.get("retry-after", MAX_INTERVAL))
                await asyncio.sleep(retry_after)
                continue
            case _:
                response.raise_for_status()
                changed = False

        interval = MIN_INTERVAL if changed else min(interval * BACKOFF_FACTOR, MAX_INTERVAL)
        await asyncio.sleep(interval)

The body hash matters because some providers rotate weak ETags on every response even when nothing changed. Hashing the payload yourself means you only pay for downstream writes when the content genuinely differs. Cache the last digest in Redis rather than process memory so a restart does not trigger a full re-sync — see caching Python API responses with Redis. If the provider has no usable API at all, the trade-offs shift entirely and web scraping vs official APIs is the honest next stop.

When to Choose Each

Avoid webhooks while you are still on a laptop with no stable public URL, when the provider's retry policy is weaker than your uptime, or when you cannot yet run a queue. Avoid polling once you cross roughly 50 watched resources at sub-five-minute freshness — that is where the vCPU bill and the rate-limit fights start compounding.

Migration Path from Polling to Webhooks

Switching is four steps and none of them require downtime. First, deploy the receiver and store raw events without acting on them, while the poller keeps running as the source of truth. Second, compare for a week: log every event id the webhook saw and every change the poller found, and alert on divergence. You want the webhook set to be a superset. Third, flip the write path so the queue worker applies events and the poller only reconciles. Fourth, stretch the poll interval from one minute to one hour and watch your request graph fall off a cliff. Keep the sweep forever. Instrument each stage through monitoring and logging Python APIs so the divergence alert is real rather than aspirational.

Builder Verdict

Webhooks win, and it is not close. For any integration where resources change less often than your freshness target — payments, CRM updates, document processing, order sync — a signed receiver plus an hourly reconciliation sweep costs a rounding error compared to polling, delivers single-digit-second latency instead of minutes, and stops you fighting rate limits at exactly the moment your customer count grows. The one honest cost is a day of extra work: signature verification, an idempotent write path, and a queue. Pay it. Polling is the right call only in two narrow cases — fewer than twenty watched resources, where operational simplicity genuinely beats compute savings, and genuinely high-frequency feeds where one request batches dozens of changes. Everywhere else, keep polling only as the slow sweep that proves your webhook stream did not lie to you.

FAQ

At what customer count does polling stop being profitable? Around 50 connected accounts at one-minute freshness. That is 2.16 million outbound requests a month and roughly 72 CPU-hours, which is where a hobby-tier instance starts throttling and you upgrade. Below 20 accounts the compute is genuinely free and the extra webhook infrastructure is not worth your time.

Does a reconciliation sweep undo the cost savings of webhooks? No. An hourly sweep across 500 accounts is 360,000 requests a month against 21.6 million for one-minute polling — a 98% reduction while still guaranteeing completeness. Move it to nightly and it drops to 15,000. The sweep is insurance you can afford.

What happens to my webhook endpoint when I rotate the signing secret? Accept both the old and new secret for one overlap window, typically 24 hours, by trying each in turn during verification. Register the new endpoint secret with the provider first, confirm live traffic verifies against it, then drop the old one. Never rotate in a single deploy — in-flight retries carry the old signature.

Is migrating from polling to webhooks risky for existing customers? Not if you shadow first. Run the receiver in log-only mode alongside the poller for a week and compare the two event streams. Customers see nothing until the sets match, and you keep the poller as a sweep, so the worst regression is a return to your current latency rather than lost data.

How do I handle a provider that fires thousands of webhooks in one burst? Return 202 immediately and absorb the burst in the queue, never in the request handler. Size the queue for peak and the workers for average, since events are not time-critical once persisted. If the provider supports it, subscribe to only the event types you act on — dropping unused types often cuts inbound volume by half.

Related

• Understanding REST vs GraphQL — the parent guide, for choosing the protocol before you choose who initiates the call.
• Processing webhooks with Python — the full receiver, queue, and replay stack once you have committed to webhooks.
• Building an idempotent webhook receiver — how to make duplicate deliveries harmless instead of corrupting data.
• Retrying failed HTTP requests with tenacity — the retry policy your reconciliation sweep needs when the provider wobbles.
• APScheduler vs Celery beat — picking the scheduler that fires the sweep without duplicating it across replicas.