← Master Index
Vol. 15 Module 15.2 Lecture

Working Memory

Agent Memory Types (added)

How This Lesson Fits the Module & Volume

Episodic and semantic stores live mostly outside the model. Working memory is what the agent can attend to right now: the context window, scratchpad, ReAct trace, and graph state. It is the bottleneck that makes retrieval, summarization, and planning necessary.

LangGraph makes working memory explicit as typed state; agent loops grow it every tool call. The next lecture, long-term memory, is what you persist when this scratchpad is gone.

Learning Objectives

By the end of this lesson, students should be able to:

  • Define working memory as capacity-limited, task-local state in the current run.
  • Map working memory onto context windows, scratchpads, and graph state objects.
  • Explain why tool results and RAG chunks compete for the same token budget.
  • Apply compaction: summarize, drop, or offload to long-term/episodic stores.
  • Contrast ReAct scratch vs LangGraph state vs multi-agent message buffers.
  • Avoid the “infinite chat log” anti-pattern.
Definition

Working memory is the agent’s short-lived workspace for the active goal: messages, thoughts, tool observations, retrieved snippets, and intermediate artifacts that must fit in the model’s attention (or an equivalent state object) to influence the next action.

The Token Budget Is the Architecture

Every extra tool observation, RAG chunk, and chat turn consumes working memory. When the window fills, the model forgets early constraints, repeats tools, or ignores HITL instructions. Treat the prompt as RAM, not a warehouse. Long-term and episodic stores are disk; retrieval is a page fault.

Working-memory slotRole in the loopOverflow tactic
System + goalHard constraintsNever drop; pin at top
Scratch / thoughtsReAct / plan notesSummarize every N steps
Tool observationsFresh evidenceKeep last k; archive rest
Retrieved semanticsFacts for this stepRe-retrieve; don’t accumulate
HITL messagesHuman correctionsPin until goal changes

Three Implementations You Will See

Context Window

  • Raw messages to the LLM
  • Simplest; opaque to code
  • Hard token cliff

Scratchpad Object

  • Structured fields in code
  • You choose what to serialize
  • Used by ReAct agents

Graph State

  • Typed dict / reducer
  • LangGraph checkpoints
  • Resumable + HITL interrupts

Compaction Sketch

A production single-agent loop should compact working memory before each LLM call. The pattern below keeps pinned constraints, a rolling recap, and only the latest observations.

from dataclasses import dataclass, field MAX_OBS = 3 MAX_SCRATCH_CHARS = 2000 @dataclass class WorkingMemory: goal: str constraints: list[str] recap: str = "" observations: list[str] = field(default_factory=list) hitl: list[str] = field(default_factory=list) def add_observation(self, text: str) -> None: self.observations.append(text) extra = self.observations[:-MAX_OBS] if extra: self.recap = (self.recap + " " + " ".join(extra)).strip()[:MAX_SCRATCH_CHARS] self.observations = self.observations[-MAX_OBS:] def to_prompt(self) -> str: parts = [ f"GOAL: {self.goal}", "CONSTRAINTS:\n- " + "\n- ".join(self.constraints), ] if self.hitl: parts.append("HUMAN:\n- " + "\n- ".join(self.hitl)) if self.recap: parts.append(f"RECAP: {self.recap}") if self.observations: parts.append("RECENT OBS:\n" + "\n".join(self.observations)) return "\n\n".join(parts) wm = WorkingMemory( goal="Enable SSO for contractor", constraints=["Do not change billing", "Cite Okta policy"], ) wm.add_observation("Ticket #881: contractor not in okta-admins") wm.add_observation("Policy chunk: contractors need sponsor approval") print(wm.to_prompt())

Multi-Agent Working Memory

Multi-agent systems multiply scratchpads: each role has a buffer, plus a shared blackboard. CrewAI and AutoGen often leak entire chat histories between agents—that is working-memory explosion, not collaboration. Prefer passing artifacts (a research brief, a schema, a recap) rather than full transcripts.

Strengths

  • Immediate control over next action
  • Explicit state enables HITL resume
  • Compaction is a measurable skill
  • Maps cleanly to graph reducers

Tradeoffs

  • Hard capacity (tokens / RAM)
  • Over-summarization loses details
  • Hidden dumps in frameworks
  • Vanishes when the process dies unless checkpointed
Common Misconception

“A 128k context means we do not need memory architecture.” Long windows still degrade attention, raise cost/latency, and mix stale tool junk with fresh goals. Working memory is a design problem: what is pinned, what is summarized, what is offloaded to long-term stores.

Knowledge Check

  1. Short Answer: What is working memory for an LLM agent? Answer: The capacity-limited live workspace (prompt/state) for the current goal.
  2. True/False: Chat history unbounded is a valid working-memory strategy. Answer: False—it is an anti-pattern.
  3. Multiple Choice: LangGraph working memory is typically: (a) typed graph state, (b) a CSS theme, (c) GPU firmware. Answer: (a).
  4. Short Answer: Name one overflow tactic for tool observations. Answer: Keep last k and summarize/archive the rest.
  5. True/False: HITL corrections should usually be pinned in working memory. Answer: True.
  6. Multiple Choice: Semantic RAG chunks in the prompt are: (a) long-term disk only, (b) working memory once retrieved, (c) episodic by definition. Answer: (b).
  7. Short Answer: Why do multi-agent chats explode working memory? Answer: Full transcripts are copied between agents instead of compact artifacts.
  8. True/False: A large context window removes the need to retrieve from long-term stores. Answer: False.
  9. Multiple Choice: Compaction happens: (a) after training CNNs, (b) before/during LLM calls in the loop, (c) only at deploy time. Answer: (b).
  10. Short Answer: What persists after working memory is gone? Answer: Long-term memory (and written episodes/semantics).

Key Takeaways

  • Working memory is the live, limited scratchpad: prompt, ReAct trace, or graph state.
  • Pin goals/constraints/HITL; compact observations; re-retrieve facts.
  • Long context ≠ infinite attention or free cost.
  • Multi-agent designs must pass artifacts, not raw transcripts.
  • Continue with Long-Term Memory.
Trainer’s Guide

Lab: Run a 12-step dummy tool loop with and without compaction; plot prompt tokens and whether the model still recites the original constraints.

Whiteboard: RAM vs disk: working vs long-term/episodic/semantic. Mark checkpoint = snapshot of working memory for HITL resume.

Recap: Working memory is scarce attention. Continue with Long-Term Memory.