Memory — Module 15.1

How This Lesson Fits the Module & Volume

An AI agent that only sees the current prompt is amnesiac. Memory is how past observations, decisions, and user facts survive across steps—and across sessions. This lecture gives the engineering map; Module 15.2 deepens the types: episodic, semantic, working, and long-term.

Vol. 14 RAG already is a form of semantic memory (documents in a vector store). Agent memory also includes scratchpads, episode logs, and user profiles. LangGraph checkpointers persist graph state; they are not a full memory architecture by themselves.

Learning Objectives

By the end of this lesson, students should be able to:

Explain why the context window is working memory, not a database.
Classify memories as working, episodic, semantic, or long-term.
Design a minimal store: scratchpad + episode log + retrievable facts.
Avoid dumping entire chat history into every call.
Connect RAG (Vol. 14) to semantic memory for agents.
Preview how Module 15.2 and MCP resources extend memory.

Definition

Agent memory is any state, outside a single model forward pass, that the agent can write and later read to improve decisions—including the in-context scratchpad, conversation summaries, episode traces, and retrieved knowledge. Memory is a system design, not a single vector index.

Four Buckets (Preview of Module 15.2)

Type	What it holds	Typical store	15.2 lecture
Working	Current goal, last tools, scratchpad	Context window / state dict	Working memory
Episodic	What happened in a run/session	Logs, traces, event store	Episodic memory
Semantic	Facts, docs, policies	Vector DB + metadata (Vol. 14)	Semantic memory
Long-term	Durable user/org knowledge	DB + embeddings + policies	Long-term memory

Working Memory Is Finite

Every tool result you append to messages consumes tokens. Naive agents “remember” by never truncating, then hit context limits, latency, and cost. A healthier pattern: keep a compact state object, summarize old turns, and retrieve only relevant long-term facts (RAG as a tool).

from dataclasses import dataclass, field
from typing import Any

@dataclass
class AgentMemory:
    goal: str
    scratch: dict[str, Any] = field(default_factory=dict)
    episodes: list[str] = field(default_factory=list)
    facts: dict[str, str] = field(default_factory=dict)  # stand-in for semantic/long-term

    def note_tool(self, name: str, observation: str) -> None:
        self.scratch["last_tool"] = name
        self.scratch["last_obs"] = observation[:500]
        self.episodes.append(f"{name}: {observation[:200]}")
        if len(self.episodes) > 20:
            self.episodes = self.episodes[-20:]

    def remember_fact(self, key: str, value: str) -> None:
        self.facts[key] = value

    def working_prompt(self) -> str:
        facts = "; ".join(f"{k}={v}" for k, v in self.facts.items()) or "(none)"
        recent = " | ".join(self.episodes[-5:]) or "(none)"
        return (
            f"Goal: {self.goal}\n"
            f"Known facts: {facts}\n"
            f"Recent episodes: {recent}\n"
            f"Scratch: {self.scratch}"
        )

mem = AgentMemory(goal="Resolve refund for ORD-1042")
mem.note_tool("get_order", "Shipped Friday; paid $80")
mem.remember_fact("preferred_channel", "email")
print(mem.working_prompt())

In production, facts becomes a vector/SQL store; episodes become structured traces; scratch may live in LangGraph state. Module 15.3 MCP resources can expose memory stores as readable resources rather than ad-hoc tools.

Memory vs RAG vs Checkpoints

RAG (Vol. 14)

Mostly read-only corpus
Great for policies/docs
Not automatically user-specific

Agent memory

Read/write during the loop
User + run specific
Needs privacy & TTL

Checkpointers

Resume a graph run
Durability, not recall quality
Complement memory, not replace it

Strengths of explicit memory

Shorter prompts, lower cost
Personalization that RAG lacks
Better multi-session agents
Auditable writes

Risks

Stale or wrong memories
PII retention / leakage
Memory poisoning via tools
Over-retrieval noise

Common Misconception

“Memory means concatenating the entire chat into every request.” That is an unbounded scratchpad, not an architecture. Real memory has write policies, retrieval, summarization, TTLs, and separation of working vs long-term stores—the subject of Module 15.2.

Knowledge Check

Short Answer: Why is the context window only working memory? Answer: It is finite, ephemeral per call, and expensive to stuff with full history.
True/False: Vol. 14 RAG is closest to semantic memory. Answer: True.
Multiple Choice: Episodic memory mainly stores: (a) CSS, (b) what happened in a run/session, (c) GPU clocks. Answer: (b).
Short Answer: Name the four Module 15.2 memory types. Answer: Episodic, semantic, working, long-term.
True/False: LangGraph checkpointers are a complete memory product. Answer: False—they persist run state, not recall quality.
Multiple Choice: A good working prompt typically includes: (a) full raw logs forever, (b) goal + compact facts + recent episodes, (c) the training corpus. Answer: (b).
Short Answer: Name one memory risk. Answer: Stale facts, PII leakage, poisoning, or over-retrieval (any).
Short Answer: How can MCP help memory later? Answer: Expose stores as MCP resources/tools with a standard protocol.
Multiple Choice: User preference “email only” belongs mainly in: (a) long-term/profile facts, (b) CNN weights, (c) stride. Answer: (a).
True/False: Agents should write every token of every tool result into long-term memory. Answer: False—write selectively with policy.

Key Takeaways

Memory is a multi-store design: working scratch + episodes + semantic/long-term facts.
Do not confuse RAG, chat history dumps, and checkpointers—they solve different jobs.
Retrieve and summarize; never unbounded-append by default.
Module 15.2 specializes each memory type; treat this lecture as the map.
Next: reflection—using traces to critique and improve.

Trainer’s Guide

Whiteboard: For a multi-day onboarding agent, list one write and one read for each of the four memory types.

Lab: Add TTL: drop facts older than 30 days (add timestamps). Show how a stale refund policy would mis-route the agent.

Recap: Memory is structured state plus retrieval—not infinite chat logs. Continue with Reflection.