Calculating Cost per API Request

You cannot defend a price you cannot cost. Most indie API builders pick a number that feels right — $29 a month, 10,000 calls included — then discover six months later that the heaviest 5% of traffic eats the entire gross margin. This page resolves that: it builds a real unit-economics model for a Python API, component by component, and turns it into a price floor you can point at. Part of the Designing API Pricing Tiers guide, it assumes you have already picked a billing axis and now need the arithmetic underneath it. The working example throughout is a document-enrichment endpoint: it validates input, hits Postgres, calls a hosted language model, and returns JSON. Every number below is a real one you can substitute with your own rate card.

The five things a request actually costs you

Variable cost per request splits into five buckets, and they differ in size by three orders of magnitude. Getting the ranking right matters more than getting any single number precise.

Compute is the cheapest and the one builders overestimate. A 1 vCPU / 2 GB container on a managed host runs about $25 a month. Serve two million requests through it and compute costs $0.0000125 per request — 12.5 micro-dollars. Compute only becomes interesting when your workers sit idle; the fix is worker sizing, not pricing, and it is covered in Uvicorn vs Gunicorn worker configuration.

Egress is almost always noise for a JSON API. An 8 KB response at $0.09/GB costs 0.7 micro-dollars. It stops being noise the moment you return files, images, or streamed payloads — at 2 MB per response the same rate costs 180 micro-dollars, and egress jumps from fifth place to second.

Database covers the managed Postgres instance plus the write amplification of your own metering. A $50/month instance across two million requests is 25 micro-dollars per call. If you write a usage row on every request — and you should, per logging API usage events to Postgres — that row is part of the cost of the request, not overhead.

Third-party and model spend dominates everything else the instant an external vendor is in the path. Our example call sends 900 input tokens and returns 300 at $0.25 and $1.25 per million respectively: 600 micro-dollars. That is 48 times the compute cost.

Observability — structured logs, traces, metric cardinality — runs about 3 micro-dollars per request at 2 KB of log volume. Small, but log vendors bill on ingest, so a debug-level deploy can multiply it tenfold overnight.

The lesson is blunt: if a vendor sits in your request path, your unit economics are their unit economics plus a rounding error. Optimizing container size while a language model burns 94% of the bill is theatre. Cut token spend first, using the tactics in controlling LLM API costs in production, then worry about the infrastructure line.

Building the model in code

Keep the rate card out of your Python. Rates change when you switch regions, renegotiate, or swap models, and you want that change to be a config edit reviewed in a pull request, not a code change. Load it from TOML, describe a request's resource shape as a frozen dataclass, and let one function price each component.

import os
import tomllib
from dataclasses import dataclass, asdict

RATE_CARD_PATH = os.getenv("RATE_CARD_PATH", "rates.toml")


@dataclass(frozen=True)
class RequestShape:
    """Resources consumed by a single request."""
    cpu_ms: float
    egress_kb: float
    db_seconds: float
    input_tokens: int
    output_tokens: int
    log_kb: float


def load_rates(path: str | None = None) -> dict[str, float]:
    with open(path or RATE_CARD_PATH, "rb") as fh:
        return tomllib.load(fh)["rates"]


def component_cost(kind: str, qty: float, rates: dict[str, float]) -> float:
    """Cost in USD for one metered component of a request."""
    match kind:
        case "cpu_ms":
            return qty / 3_600_000 * rates["compute_usd_per_vcpu_hour"]
        case "egress_kb" | "log_kb":
            key = "egress_usd_per_gb" if kind == "egress_kb" else "log_usd_per_gb"
            return qty / 1_048_576 * rates[key]
        case "db_seconds":
            return qty / 3600 * rates["db_usd_per_hour"]
        case "input_tokens":
            return qty / 1_000_000 * rates["model_usd_per_m_input"]
        case "output_tokens":
            return qty / 1_000_000 * rates["model_usd_per_m_output"]
        case _:
            raise ValueError(f"unknown cost component: {kind}")


def cost_of(shape: RequestShape, rates: dict[str, float]) -> dict[str, float]:
    breakdown = {k: component_cost(k, v, rates) for k, v in asdict(shape).items()}
    breakdown["total"] = sum(breakdown.values())
    return breakdown


if __name__ == "__main__":
    mean_request = RequestShape(
        cpu_ms=float(os.getenv("MEAN_CPU_MS", "45")),
        egress_kb=float(os.getenv("MEAN_EGRESS_KB", "8")),
        db_seconds=float(os.getenv("MEAN_DB_SECONDS", "1.8")),
        input_tokens=int(os.getenv("MEAN_INPUT_TOKENS", "900")),
        output_tokens=int(os.getenv("MEAN_OUTPUT_TOKENS", "300")),
        log_kb=float(os.getenv("MEAN_LOG_KB", "2")),
    )
    for name, usd in cost_of(mean_request, load_rates()).items():
        print(f"{name:>14}: {usd * 1_000_000:8.1f} micro-dollars")

The db_seconds field deserves a note. You are renting a whole instance, so the honest allocation is the fraction of that instance one request occupies — query time, not query count. Measure it once with your pool's checkout timer and treat it as fixed until your schema changes. If your pool is saturated the number lies badly, which is one more reason to keep an eye on connection pool exhaustion.

The mean is a marketing number, the p95 is the bill

Averages hide the customers that hurt you. In the enrichment example, the mean request costs 641 micro-dollars but the 95th-percentile request costs 2,058 — a 3.2x spread — because long documents burn more tokens, miss the cache, and hold a database connection longer. Three multipliers stack on the same request, and they are correlated rather than independent.

That spread would be harmless if every customer drew randomly from the distribution. They do not. Customer usage is lumpy and self-similar: one integration sends tidy 400-token snippets all day, another pipes in scanned PDFs and lives permanently in the tail. Price against the mean and the tail customer is served at a loss, quietly, on the same tier as everybody else.

So compute both, per customer, and act on the ratio. The snippet below reads priced usage rows and reports the mean, p95, and the tail multiplier for every account. Anything above 2x deserves a look; anything above 4x is a pricing conversation or a rate limit.

import os
import statistics
from collections import defaultdict
from collections.abc import Iterable

TAIL_ALERT_RATIO = float(os.getenv("TAIL_ALERT_RATIO", "2.0"))


def cost_profile(rows: Iterable[tuple[str, float]]) -> dict[str, dict[str, float]]:
    """rows: (customer_id, request_cost_usd). Returns mean, p95 and tail ratio."""
    by_customer: dict[str, list[float]] = defaultdict(list)
    for customer_id, usd in rows:
        by_customer[customer_id].append(usd)

    profile = {}
    for customer_id, costs in by_customer.items():
        costs.sort()
        mean = statistics.fmean(costs)
        p95 = (
            statistics.quantiles(costs, n=20)[-1] if len(costs) >= 20 else costs[-1]
        )
        profile[customer_id] = {
            "requests": len(costs),
            "mean_usd": mean,
            "p95_usd": p95,
            "tail_ratio": p95 / mean if mean else 0.0,
            "flagged": (p95 / mean if mean else 0.0) > TAIL_ALERT_RATIO,
        }
    return profile

Run this weekly against your usage table and surface it in a customer usage dashboard so support sees the same number you do.

The gross-margin floor

Now turn cost into a minimum price. Four adjustments separate raw cost from a list price, and skipping any one of them is how builders end up at 40% gross margin wondering where the money went.

Start with the mean billable cost of 641 micro-dollars. First, divide by the billable share of traffic. If 8% of requests come from free-tier accounts that never convert, paying customers carry that load: 641 ÷ 0.92 = 697. Second, divide by your target cost share. An 80% gross margin means cost is 20% of price, so 697 ÷ 0.20 = 3,483 micro-dollars. Third, add payment processing: 2.9% of the charge plus a fixed 30 cents per invoice, which across a 40,000-request average invoice adds 7.5 micro-dollars per request. Solving for the gross price gives (3,483 + 7.5) ÷ 0.971 = 3,594. Fourth, round up and give yourself headroom for the tail: $0.004 per request, or $4 per thousand.

Translate that into a package and the tiers write themselves: a $29 tier with 8,000 requests included prices each call at 3,625 micro-dollars — above the floor — while a $99 tier with 30,000 included sits at 3,300 and needs either a volume-discount justification or a smaller allowance. That is the whole point of the floor. It converts "does this tier feel cheap?" into a yes-or-no test. Wire the resulting per-unit price into Stripe metered billing configuration and the model stops being a spreadsheet and starts being an invoice.

When to model per request — and when not to

Per-request modelling earns its keep when a variable, per-call vendor cost dominates: model tokens, geocoding lookups, SMS sends, enrichment providers. There the marginal request has a real price tag and margin genuinely moves with volume.

It is the wrong lens when your bill is almost entirely fixed. A CRUD API on a $25 container and a $7 database has a marginal cost near zero and a fixed cost that only steps when you add an instance. Model that per tenant-month instead, and spend your effort on the step function — the traffic level at which you add a machine — rather than on six decimal places per call. Hosting choice shifts that step function considerably, which is what Render vs Railway vs Fly.io compares.

Skip the exercise entirely below roughly 50,000 requests a month. At that volume the entire infrastructure bill is smaller than one hour of your time, and the honest answer is to ship features. Start modelling when a vendor invoice grows faster than revenue, when you are about to publish a public price, or when one account's traffic exceeds 20% of the total.

The transition path: instrumenting cost attribution

Moving from guesswork to a live model takes four steps and about an afternoon.

Step one: attach a mutable cost accumulator to every request and let handlers add to it. Step two: price it in middleware and emit the total on the access log line. Step three: persist that number alongside the usage row you already write. Step four: replace the estimated rate card with the real numbers from last month's invoices, divided by last month's request count. Reconciliation is the step people skip, and it is the one that catches the storage line you forgot.

import os
import time
from contextvars import ContextVar
from fastapi import FastAPI, Request

from cost_model import RequestShape, cost_of, load_rates

RATES = load_rates()
usage: ContextVar[dict[str, float]] = ContextVar("usage")
app = FastAPI()


@app.middleware("http")
async def attribute_cost(request: Request, call_next):
    token = usage.set({"input_tokens": 0.0, "output_tokens": 0.0, "db_seconds": 0.0})
    started = time.perf_counter()
    try:
        response = await call_next(request)
    finally:
        elapsed_ms = (time.perf_counter() - started) * 1000
        counters = usage.get()
        shape = RequestShape(
            cpu_ms=elapsed_ms,
            egress_kb=float(response.headers.get("content-length", 0)) / 1024,
            db_seconds=counters["db_seconds"],
            input_tokens=int(counters["input_tokens"]),
            output_tokens=int(counters["output_tokens"]),
            log_kb=float(os.getenv("LOG_KB_PER_REQUEST", "2")),
        )
        response.headers["x-request-cost-micro"] = f"{cost_of(shape, RATES)['total'] * 1e6:.1f}"
        usage.reset(token)
    return response

Handlers add to the counters with usage.get()["input_tokens"] += n after each model call. Emit the total through structured logging with structlog rather than a response header in production — the header above is for local inspection.

Builder verdict

Build the model, but build the cheap version. A frozen dataclass, a TOML rate card, and a middleware that stamps a cost on every request will get you within 10% of truth in an afternoon, and 10% is plenty when you are deciding between $3 and $4 per thousand calls. The winner here is the per-request model priced off the mean with a p95 alert on top — not a per-tenant allocation, and not a full activity-based costing exercise. Set your floor at mean cost divided by your target cost share, adjusted for free-tier drag and card fees, then round up. Where cost really bites is the tail, and the answer to the tail is operational, not arithmetic: cache aggressively with Redis response caching, cap request size, and rate-limit the accounts your tail-ratio report flags. Price on the mean, defend the p95, and revisit the rate card every quarter.

FAQ

What gross margin should an API side project target? Aim for 80% at the mean request. Below 70% you have no room for support, refunds, or a bad month of vendor pricing, and below 50% you are reselling somebody else's infrastructure at a markup. If a vendor call makes 80% impossible, the fix is caching, smaller models, or a higher price — not accepting the margin.

Should I price on the mean cost or the p95 cost? Price on the mean, monitor the p95. Pricing on the p95 makes you uncompetitive for the 95% of traffic that is cheap. Instead, set the floor from the mean and control the tail with request-size caps, per-account rate limits, and a tail-ratio alert that flags accounts costing more than twice their mean.

How do free-tier users change the calculation? They raise the effective cost of every paid request. Divide your mean cost by the share of traffic that is billable — at 8% free traffic that is a 9% cost increase carried by paying customers. If free traffic climbs past 20% of volume, tighten the tier using the tactics in the free-tier abuse guide rather than raising prices.

What is the migration risk when I raise prices after modelling costs? Grandfather existing customers for at least one full billing cycle and announce the new price 30 days ahead. Most churn at a price change comes from surprise, not from the number. Publishing the included allowance and the overage rate up front also protects you when a vendor raises rates and you need to follow.

How often should I refresh the rate card? Quarterly, and immediately after any vendor pricing change or region move. Reconcile the modelled cost against the actual invoice each month: divide the total bill by the request count and compare. A drift above 15% means a component is missing from the model, usually storage, backups, or log ingest.

Related

Same track:

• Designing API Pricing Tiers — the parent guide that turns this cost floor into published tiers.
• Usage-Based vs Seat-Based Pricing — pick the billing axis before you compute a per-unit floor.
• Preventing Free-Tier Abuse — keeps the unbillable share of traffic small enough to absorb.

Other tracks:

• Logging API Usage Events to Postgres — the usage table this model reads its per-request costs from.
• Controlling LLM API Costs in Production — cut the component that dominates 94% of the bill.