The $2K/month cache: a high-usage Redis story

Every cost story starts the same way: a line item gets big enough that someone in finance asks a question engineering can't answer in one sentence. Ours was Redis. The courier-assignment platform leaned on it for a hot lookup on every request — at ~25k req/s, that adds up — and the monthly bill had crept past the point where “it's just cache” was an acceptable explanation.

The instinct in the room was to scale the cluster: more memory, more replicas, a bigger tier. That's the expensive answer. The cheaper one was hiding in the access pattern.

Look at the keys before you look at the cluster

Before touching infrastructure, I pulled a day of cache traffic and counted distinct keys on the dimension we were caching — the company identifier attached to each shipment. The number surprised everyone who hadn't looked:

Eight hundred. Not eight hundred thousand. The entire keyspace on that dimension fit comfortably in a few megabytes of process memory — and it changed slowly, on the order of new companies onboarding, not per-request. We were paying network round-trips and a managed-service premium to look up a dataset small enough to keep on the box.

Low cardinality is a gift. When the set of things you cache is small and slow-moving, the network hop is pure overhead.

The change: an in-memory LRU in front of Redis

The fix was a small, bounded in-process cache that sits in front of Redis and absorbs the reads. Redis stays as the shared source of truth and handles the long tail; the LRU handles the 80%+ of traffic that hammers the same few hundred keys.

// bounded in-process cache, refreshed lazily with a short TTL
cache := lru.New[string, *Company](2048)

func GetCompany(ctx context.Context, id string) (*Company, error) {
    if c, ok := cache.Get(id); ok {
        return c, nil   // served from RAM, ~0 network
    }
    c, err := redisGet(ctx, id)  // miss → shared cache
    if err == nil {
        cache.Add(id, c)
    }
    return c, err
}

Two things mattered for correctness. First, the LRU is bounded — 2,048 entries caps memory even if the keyspace grows, and the eviction policy keeps the hot set resident. Second, a short TTL on entries means a company record update propagates within seconds without any explicit invalidation plumbing. The semantics didn't change; only the location of the read did.

What we measured

• Redis traffic fell ~80%. The shared cache went from “every request” to “cold key or post-TTL refresh.”
• Response time dropped ~40ms on the cached path — a RAM read instead of a network round-trip.
• ~$2K/month reclaimed. We dropped to a smaller Redis tier because the load no longer justified the big one.

Why this generalizes

The specific numbers are ours, but the move isn't. Caching is usually framed as a latency tool, so we reach for it without asking what the data actually looks like. The questions that found the win here are cheap to ask anywhere:

• How many distinct values does this key actually take?
• How often does each value change?
• What am I paying — in dollars and milliseconds — for the round-trip to fetch it?

When the answer is “few, rarely, and more than you'd think,” the cheapest cache is the one closest to the CPU. No new cluster required — just a look at the keys before the dashboard.