Redis vs In-Memory Caching for FastAPI

A process-local dictionary is the fastest cache you will ever ship: no network hop, no serialization, no extra bill. It is also the cache that quietly breaks the moment you scale past one container. This page resolves that choice for a FastAPI service — when a plain in-process cache is genuinely the right call, when shared Redis earns its round trip, and why the answer for most commercial APIs is both, layered. It extends the parent guide on caching Python API responses with Redis, which covers key design, TTLs, and stampede locks you will reuse here.

The decision hinges on three numbers you can actually measure: your replica count, your working-set size, and how much staleness a customer will tolerate before they file a ticket. Everything else is detail.

Process-local caches versus a shared Redis cache Left panel shows a load balancer fanning out to four web replicas, each holding its own private cache copy. Right panel shows the same four replicas sharing one Redis instance. Process-local cache Load balancer web 1 cache web 2 cache web 3 cache web 4 cache 4 copies, 4 cold starts a write clears 1 of 4 hit rate 80% at 4 replicas read latency 0.01 ms Shared Redis Load balancer web 1 web 2 web 3 web 4 Redis, one copy hit rate 95% at any replica count read latency 1.2 ms

The comparison that actually matters

Most write-ups compare these two on raw latency, which is the least interesting axis. A Python dict lookup costs microseconds and a same-region Redis GET costs roughly one millisecond — both vanish next to the 30–60 ms database query you are avoiding. The axes that change your architecture are hit rate under horizontal scaling, invalidation correctness, and RAM billed per instance.

AxisProcess-local dictShared Redis
Read latency~0.01 ms~1.2 ms same region
Hit rate at 8 replicasDegrades sharplyUnchanged
Cross-replica invalidationImpossible without pub/subOne DEL
RAM costDuplicated per instanceOne copy, billed once
Survives a deployNo — cold every restartYes

The first row is the only one where the dict wins, and it wins by an amount your customers cannot perceive. Every other row is a structural advantage for Redis that compounds as you add capacity.

Hit rate is a function of replica count

Here is the arithmetic that surprises people. Suppose your hot working set is 500 distinct cache keys and you serve 10,000 requests inside one TTL window. With a shared cache, you miss once per key — 500 misses, a 95% hit rate. With process-local caches and round-robin routing, every replica must independently miss on every key before it warms up. Four replicas means 2,000 misses. Eight replicas means 4,000 misses, and your hit rate collapses to 60% while your database load quadruples.

Cache hit rate versus replica count for a process-local cache Bar chart showing hit rate falling from 95 percent at one replica to 80, 60 and 40 percent at four, eight and twelve replicas, against a flat 95 percent reference line for shared Redis. Local-cache hit rate: 500 keys, 10,000 requests per TTL window 0% 25% 50% 75% 100% Redis stays at 95% however many replicas 95% 1 replica 500 misses 80% 4 replicas 2,000 misses 60% 8 replicas 4,000 misses 40% 12 replicas 6,000 misses

That is the trap: the in-memory cache looks brilliant in staging with one container and degrades exactly when traffic forces you to scale out. Worse, the degradation hides in your average latency — p50 still looks fine — while your database connection pool and your p99 quietly go to pieces. If you have ever chased connection pool exhaustion after a scale-up, a fragmented local cache is a prime suspect.

In-memory caching in FastAPI, done properly

A local cache is not just @lru_cache. That decorator has no TTL, no size accounting for async values, and it will happily pin stale objects forever. Write a small TTL cache instead, guard it with an asyncio.Lock so a hundred concurrent requests do not all recompute the same key, and cap its size so one weird query cannot exhaust your container's RAM.

Python
import asyncio
import os
import time
from collections import OrderedDict
from typing import Any, Awaitable, Callable

LOCAL_TTL = float(os.getenv("LOCAL_CACHE_TTL_SECONDS", "5"))
LOCAL_MAX_ENTRIES = int(os.getenv("LOCAL_CACHE_MAX_ENTRIES", "2000"))


class TTLCache:
    """Bounded, TTL-aware, single-process cache safe for asyncio."""

    def __init__(self, ttl: float, max_entries: int) -> None:
        self._ttl = ttl
        self._max = max_entries
        self._data: OrderedDict[str, tuple[float, Any]] = OrderedDict()
        self._locks: dict[str, asyncio.Lock] = {}

    def get(self, key: str) -> Any | None:
        entry = self._data.get(key)
        if entry is None:
            return None
        expires_at, value = entry
        if time.monotonic() >= expires_at:
            self._data.pop(key, None)
            return None
        self._data.move_to_end(key)
        return value

    def set(self, key: str, value: Any) -> None:
        self._data[key] = (time.monotonic() + self._ttl, value)
        self._data.move_to_end(key)
        while len(self._data) > self._max:
            self._data.popitem(last=False)

    async def get_or_load(
        self, key: str, loader: Callable[[], Awaitable[Any]]
    ) -> Any:
        hit = self.get(key)
        if hit is not None:
            return hit
        lock = self._locks.setdefault(key, asyncio.Lock())
        async with lock:
            hit = self.get(key)          # another task may have filled it
            if hit is not None:
                return hit
            value = await loader()
            self.set(key, value)
            return value


local_cache = TTLCache(LOCAL_TTL, LOCAL_MAX_ENTRIES)

Two details earn their keep. The double check inside the lock stops a stampede without every waiter recomputing. The OrderedDict with popitem(last=False) gives you least-recently-used eviction in two lines, so the cache has a hard memory ceiling you can reason about when sizing an instance.

Invalidation is where local caches actually fail

Latency and hit rate are performance problems. Invalidation is a correctness problem, and correctness problems become refunds. With a shared cache, a write deletes one key and every replica sees the change on its next read. With process-local caches, the replica that handled the write updates its own copy and the other replicas keep serving the old value until their TTL expires. Your customer updates a price, refreshes, and sees the new value — then refreshes again, lands on a different replica, and sees the old one.

Stale read sequence across two replicas with local caches Sequence diagram: a client writes to replica A which updates its own cache, then reads from replica B which still returns the old cached value until its TTL expires. Client Replica A Replica B PUT /plans/42 price = 99 writes DB, refreshes own cache GET /plans/42 routed to B 200 OK price = 79 (stale) B keeps the old value until its own TTL expires.

You can patch this with Redis pub/sub broadcasting invalidations to every process, but notice what just happened: you added Redis anyway, and you added a distributed-messaging failure mode on top of it. If a replica misses a message during a restart it serves stale data indefinitely. That is why the shared-Redis pattern and the cache invalidation strategies built on it stay simpler: one authoritative copy, one delete.

Here is the shared version, using redis.asyncio with a connection pool created once at startup:

Python
import json
import os
import redis.asyncio as redis

REDIS_URL = os.getenv("REDIS_URL", "redis://localhost:6379/0")
NAMESPACE = os.getenv("CACHE_NAMESPACE", "api:v1")
REDIS_TTL = int(os.getenv("REDIS_CACHE_TTL_SECONDS", "300"))

pool = redis.ConnectionPool.from_url(REDIS_URL, max_connections=20)


def client() -> redis.Redis:
    return redis.Redis(connection_pool=pool, decode_responses=True)


async def read_through(key: str, loader) -> dict:
    full_key = f"{NAMESPACE}:{key}"
    conn = client()
    cached = await conn.get(full_key)
    if cached is not None:
        return json.loads(cached)
    value = await loader()
    await conn.set(full_key, json.dumps(value), ex=REDIS_TTL)
    return value


async def invalidate(key: str) -> None:
    await client().delete(f"{NAMESPACE}:{key}")

One delete and the change is visible to every replica on the next request. No broadcast, no per-process bookkeeping, no divergence.

Memory cost per instance

RAM in a local cache is billed once per replica because every replica keeps a full copy. Take a realistic working set: 20,000 cached responses averaging 4 KB of serialized JSON. That is 80 MB — except Python objects carry substantial per-object overhead, so a dict of decoded structures often lands at two to three times the wire size. Call it 200 MB of real heap per container. Across eight replicas you are paying for 1.6 GB of duplicated data, and you have pushed every instance into a larger, more expensive tier.

Duplicated cache memory across replicas versus one shared copy Eight bars representing eighty megabytes of cache held by each replica, totalling six hundred and forty megabytes, next to a single block showing eighty megabytes held once in Redis. Working set: 20,000 entries at 4 KB serialized 8 replicas at 80 MB each 640 MB duplicated forces a bigger instance tier Redis: 80 MB one copy for the whole fleet 80 MB total roughly 10 dollars a month

A managed 250 MB Redis runs about ten dollars a month. Moving from a 1 GB to a 512 MB instance tier across eight replicas usually saves more than that, so shared caching frequently reduces your bill. Fold that into your cost per API request model before you decide Redis is an expense you cannot justify.

The two-tier pattern that usually wins

Now the actual recommendation. Run both, with a very short L1 in each process and a normal-length L2 in Redis. The L1 absorbs the burst of identical requests that arrive within a second or two of each other — the same product page hit forty times, the same auth lookup on every request in a session — without any network hop. The L2 keeps every replica coherent and survives restarts.

The trick is the TTL ratio. Give L1 three to five seconds and L2 five minutes. Divergence between replicas is then bounded at five seconds, which is invisible for read-mostly data, while Redis traffic on hot keys drops by an order of magnitude.

Python
from fastapi import FastAPI, Depends

app = FastAPI()


async def cached_json(key: str, loader) -> dict:
    """L1 process cache in front of the shared L2 in Redis."""
    return await local_cache.get_or_load(
        key, lambda: read_through(key, loader)
    )


async def load_plan(plan_id: int) -> dict:
    # Replace with your real async database or upstream call.
    return {"id": plan_id, "price_cents": 9900}


@app.get("/plans/{plan_id}")
async def get_plan(plan_id: int) -> dict:
    return await cached_json(
        f"plan:{plan_id}", lambda: load_plan(plan_id)
    )


@app.put("/plans/{plan_id}")
async def update_plan(plan_id: int, price_cents: int) -> dict:
    record = {"id": plan_id, "price_cents": price_cents}
    await invalidate(f"plan:{plan_id}")   # L2 clears fleet-wide
    return record
Two-tier cache request path with L1 process cache and L2 Redis A request flows into an L1 process cache serving 92 percent of traffic, then to L2 Redis serving 7 percent, then to the origin database for the remaining 1 percent, with latency labels. Where 10,000 requests are answered Request L1 process dict TTL 5 s miss L2 Redis TTL 300 s miss Database 9,200 served 0.01 ms 700 served 1.2 ms 100 served 38 ms Blended latency 0.5 ms; origin sees 1% of traffic. Worst-case replica divergence: 5 seconds.

Note the one rule you must not break: never invalidate only L1 on a write. Deleting the Redis key is what makes the change fleet-wide; the L1 copies simply age out within their five-second window. If a specific endpoint cannot tolerate even that, skip L1 for that route rather than building a broadcast layer.

When to choose each

Choose a pure in-memory cache when you run exactly one process and expect to keep running one — a scheduled job, an internal tool, a free-tier deployment on a single container. Also choose it for data that is genuinely immutable and cheap to hold: a parsed config file, a compiled regex table, a JWKS document you fetch once per hour. Static reference data has no invalidation problem, so the local cache's weakness never surfaces.

Avoid a pure in-memory cache when you run more than one replica, when your platform autoscales, when writes must be visible immediately, or when the cached payload is large enough to influence your instance tier. Any deployment on Render, Railway, or Fly.io that can scale horizontally falls here — see the Render vs Railway vs Fly.io comparison for how each platform handles multiple instances.

Choose Redis when you have two or more replicas, when the cache also carries rate-limit counters or session state, or when a cold cache after each deploy would hammer your database. Redis is also the only sane option once you need distributed locks or shared quota counters for preventing free-tier abuse.

Avoid Redis when your entire product is one container and one developer, and adding a second service would double your operational surface for no measurable gain. That is a real situation early on, and it is temporary.

The migration path from local to shared

Switching is a half-day job if you did one thing right: put the cache behind a function, not scattered @lru_cache decorators. The concrete steps:

  1. Wrap every existing local cache read in a single cached_json(key, loader) helper and deploy that change alone. Behaviour is unchanged; you have only centralized the call site.
  2. Provision Redis in the same region as your API and set REDIS_URL as a secret. Cross-region adds 30–80 ms per hit and destroys the entire benefit.
  3. Implement read_through as shown above and call it from inside the helper, keeping the local layer in front. Set LOCAL_CACHE_TTL_SECONDS=5.
  4. Add a namespace prefix from CACHE_NAMESPACE so a bad deploy can be rolled back by bumping api:v1 to api:v2 instead of flushing anything.
  5. Replace every write path's local eviction with invalidate(key), which deletes from Redis.
  6. Emit hit-rate metrics per tier before and after, so you can prove the change worked — the monitoring and logging guide covers wiring those counters into structured logs.

Do not attempt the reverse migration. Removing Redis from a multi-replica service to "save ten dollars" trades a known cost for an unbounded correctness risk.

Builder verdict

Ship the two-tier cache: Redis as the source of truth for cached data, a five-second process-local layer in front of it. Pure in-memory wins only in the single-container case, and single-container is a stage you are trying to grow out of — building on it means rewriting your cache layer exactly when you are busiest firefighting a traffic spike. Pure Redis is correct but leaves a measurable amount of latency and connection churn on the table for hot keys, and the L1 that fixes that is forty lines you write once. Commercially, the maths is one-sided: ten dollars a month for a managed Redis is less than one support ticket about stale pricing, and consolidating duplicated cache RAM usually pays for the instance downgrade that funds it. The failure mode to fear is not the Redis bill, it is discovering at 4× traffic that your hit rate is 40% and your database is the bottleneck. Write the cache behind one helper function today, and the choice stays yours to change later.

FAQ

Does adding Redis actually increase my monthly bill? Usually less than you expect, and often not at all. A 250 MB managed instance costs roughly ten dollars a month, while removing duplicated cache memory from every replica frequently lets you drop one instance tier across the fleet. Price both sides before assuming Redis is a net cost.

What happens to my API if Redis goes down? Design for it explicitly: wrap cache reads in a short timeout and treat any error as a miss, so requests fall through to the database instead of failing. A five-second local layer also keeps the hottest keys answering during a brief Redis blip, which turns an outage into a latency bump.

How much staleness does the two-tier pattern introduce? Exactly the L1 TTL — five seconds with the settings shown here. A write deletes the Redis key immediately, so every replica converges once its local copy expires. For endpoints where five seconds is unacceptable, bypass L1 on that route rather than adding a broadcast layer.

Is it risky to migrate a live API from local caching to Redis? Low risk if you keep the local layer in front and add Redis behind it, because the fallback path is unchanged. Deploy the helper refactor first, then the Redis layer, then move invalidation. Each step is independently revertible, and a namespace prefix lets you abandon a bad cache generation without flushing anything.

Should I use Redis for rate limiting and quotas too? Yes, and that is often the argument that settles the decision. Per-customer quota counters must be shared across replicas or customers get N times their allowance. Once Redis is there for metering, using it for response caching costs you nothing extra.