Redis — Module 18.2

How This Lesson Fits the Module & Volume

Your FastAPI replicas on Docker / Kubernetes are stateless. They still need a fast shared brain: rate limits, idempotency keys, session/cache, pub/sub, and—next lecture—a Celery broker. Redis is that in-memory data store. LLM calls are expensive; caching identical prompts (with care) and sliding-window rate limits are how API wrappers stay solvent.

Redis is not a vector database (Vol. 14) and not Postgres. Use it for hot, ephemeral, or queue metadata. Persistence is optional and never a substitute for your system of record.

Learning Objectives

By the end of this lesson, students should be able to:

Place Redis beside the API: cache, rate limit, broker, pub/sub.
Use keys with TTLs for prompt-response cache and idempotency.
Implement a simple token-bucket / INCR rate limit per API key.
Wire REDIS_URL from Compose/K8s without baking it into the image.
Know when not to cache (PII, tool results, rapidly changing tools).
Hand Redis to Celery as broker/backend next.

Definition

Redis (Remote Dictionary Server) is an in-memory key-value store with rich types (strings, hashes, lists, sets, sorted sets, streams) and optional persistence (RDB/AOF). It is typically single-threaded per command for simplicity and very low latency. In AI backends it is used as cache, lock, rate limiter, and message broker—not as the primary document store for chat history (use Postgres) and not as pgvector.

Jobs Redis Does in an LLM API

Pattern	Redis types	Why
Response cache	STRING + TTL	Skip duplicate completions
Idempotency	SET NX + TTL	Paid retry safety (API lecture)
Rate limit	INCR / sliding ZSET	Protect vendor quotas and your wallet
Celery broker	lists / streams (via Kombu)	Next lecture
Pub/sub or Streams	PUBLISH / XADD	Fan-out job events to WS later

Cache + Rate Limit Sketch

import hashlib, json, os
import redis
from fastapi import FastAPI, HTTPException, Request
from openai import OpenAI

r = redis.Redis.from_url(os.environ["REDIS_URL"], decode_responses=True)
client = OpenAI()
app = FastAPI()

def cache_key(prompt: str, model: str) -> str:
    h = hashlib.sha256(f"{model}|{prompt}".encode()).hexdigest()
    return f"chat:v1:{h}"

@app.post("/v1/chat")
def chat(request: Request, body: dict):
    api_key_id = request.headers.get("x-api-key-id", "anon")
    # 60 requests / 60s sliding-ish window via INCR+EXPIRE
    rl = f"rl:{api_key_id}"
    n = r.incr(rl)
    if n == 1:
        r.expire(rl, 60)
    if n > 60:
        raise HTTPException(status_code=429, detail="rate_limited")

    prompt = body["prompt"]
    model = "gpt-4.1-mini"
    ck = cache_key(prompt, model)
    hit = r.get(ck)
    if hit:
        return json.loads(hit)

    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        max_tokens=512,
    )
    out = {
        "text": resp.choices[0].message.content or "",
        "model": resp.model,
        "cached": False,
    }
    r.setex(ck, 300, json.dumps(out))  # 5 min TTL — do not cache PII/tools blindly
    return out

Cache Policy (Do Not Be Clever-Stupid)

Safe-ish to cache

FAQ / docs Q&A at temp 0
Public embeddings of public docs
Idempotency of identical POST bodies

Do not cache

User PII, medical, HR chats
Tool-using agents (world changes)
High-temperature creative gens

Ops

Compose: redis:7-alpine
K8s: StatefulSet or managed Redis
Auth Redis itself (ACL / TLS)

Why Redis

Sub-ms ops; simple mental model
One box for cache + broker + locks
Everywhere in AI boilerplate

Limits

Memory is the capacity plan
Not your chat archive
A hot key can still melt a shard

Related Lectures

Lecture	Role
Celery	Broker/result backend next
FastAPI	Where INCR/GET run
Authentication	Rate limit per identity, not per IP only
Monitoring	Cache hit ratio + 429 count

Common Misconception

“Redis is a vector DB.” Redis Stack can add vectors; this lecture means vanilla Redis for cache/queues. Use the Vol. 14 store you already chose for RAG. Second: caching all prompts will leak User A’s answer to User B if the key ignores tenant ID. Third: FLUSHALL in prod is not a deploy step. Fourth: Redis up ≠ durable chat history.

Knowledge Check

Short Answer: Name two AI-backend uses of Redis. Answer: Any two of: cache, rate limit, idempotency, Celery broker, pub/sub, locks.
True/False: Chat archives should live only in Redis forever. Answer: False—use a durable DB; Redis is hot/ephemeral.
Multiple Choice: Too many requests should return: (a) 429, (b) 301, (c) DDIM. Answer: (a).
Short Answer: Why include tenant/user id in a cache key? Answer: Prevent cross-tenant leakage of cached completions.
True/False: TTL on cached completions is a good default. Answer: True.
Multiple Choice: Celery often uses Redis as: (a) broker, (b) CFG, (c) LoRA. Answer: (a).
Short Answer: Why not cache tool-using agent turns blindly? Answer: Side effects / world state change; stale or dangerous replays.
True/False: REDIS_URL belongs in the Docker image layer. Answer: False—runtime env/config.
Multiple Choice: Next lecture: (a) Celery, (b) CLIP, (c) UMAP. Answer: (a).
Short Answer: What does SETEX give you that a bare SET does not? Answer: Automatic expiry (TTL).

Key Takeaways

Redis = shared memory for cache, limits, locks, queues.
TTL + tenant-safe keys; never cache PII/tools casually.
429 + INCR protects vendor quotas.
Not Postgres, not pgvector.
Next: Celery.

Trainer’s Guide

Lab: Compose API+Redis. Hit the same prompt twice; show cache hit. Burst 70 requests; show 429. Add tenant id to the key and demo isolation.

Whiteboard: Stateless Pods → Redis. Draw Celery workers consuming the same Redis next.

Recap: Redis is the hot shared store. Queue long work next with Celery.