AutoGen — Module 15.4

How This Lesson Fits the Module & Volume

This is the Volume 15 capstone. Vol. 14.3 AutoGen introduced conversable agents as an orchestration style beside RAG. Here we treat AutoGen as a conversation-centric multi-agent runtime: assistant + user proxy, group chat, tool/code loops, and HITL—then choose among all 15.4 frameworks using everything from Modules 15.1–15.3 (loops, memory types, MCP).

Volume 16 opens with speech: multimodal I/O on top of the agent stack you now have.

Learning Objectives

By the end of this lesson, students should be able to:

Explain AutoGen’s message-passing model (assistant, user proxy, group chat).
Configure termination and HITL (human_input_mode) on the proxy.
Treat code execution as a high-risk tool loop, not a default.
Compare AutoGen vs CrewAI vs LangGraph vs LangChain vs LlamaIndex.
Pick a framework given HITL, state, data, and multi-agent needs.
Recap how memory types and MCP attach regardless of framework.

Definition

AutoGen (Microsoft) is a framework for multi-agent LLM applications where agents coordinate by sending messages. Agents may be LLM-backed, tool-enabled, code-executing, or human proxies. Group chat managers decide who speaks next—a conversation metaphor rather than CrewAI roles or LangGraph edges.

Capstone Comparison — When to Pick Which

Framework	Metaphor	Reach for it when…	Vol. 14.3 vs 15.4
LangChain	Tool-calling / ReAct loop	Single actor, short tools, fast prototype	14.3 chains/RAG → 15.4 agent executor
LangGraph	State machine / graph	HITL, checkpoints, explicit topology	14.3 RAG critique graph → 15.4 agent control plane
LlamaIndex	Knowledge agent over indexes	Data connectors + agentic RAG	14.3 query engine → 15.4 Function/ReAct agent
CrewAI	Roles + tasks + process	SOP-style teams, artifact handoffs	14.3 crew vs RAG → 15.4 tool belts & HITL
AutoGen	Chats / group chat	Debate, code-pair, human proxy in the thread	14.3 chat intro → 15.4 HITL + tool loops

Conversation Patterns

Assistant + UserProxy

Two-party tool/code loop
Proxy can execute or ask human
Classic AutoGen starter

GroupChat

N specialists + manager
Speaker selection policy
Easy chatter / WM explosion

Nested chats

Sub-conversations for subgoals
Return a summary artifact
Closer to hierarchical crews

User Proxy, HITL, and Tool Loop

API names vary by AutoGen version (classic AssistantAgent vs newer AgentChat). The pattern is stable: an LLM agent proposes actions; a proxy runs tools/code or stops for a human; termination messages end the loop.

# Conceptual AutoGen-style pattern (classic API; names vary)
from autogen import AssistantAgent, UserProxyAgent

assistant = AssistantAgent(
    name="sso_assistant",
    system_message=(
        "You help with contractor SSO. Use lookup_ticket and policy_search. "
        "Never run shell. Propose Jira comments but wait for human approval. "
        "When done, reply exactly: TERMINATE"
    ),
    llm_config={"model": "gpt-4o-mini"},
)

user = UserProxyAgent(
    name="ops_proxy",
    human_input_mode="TERMINATE",  # ALWAYS = full HITL; NEVER = autonomous (risky)
    max_consecutive_auto_reply=6,
    code_execution_config=False,   # do not silently enable shell/Python exec
    function_map={
        "lookup_ticket": lookup_ticket,
        "policy_search": policy_search,  # semantic LTM
    },
    is_termination_msg=lambda m: "TERMINATE" in (m.get("content") or ""),
)

user.initiate_chat(
    assistant,
    message="Ticket 881: can we enable contractor SSO? Cite policy.",
)
# After chat: write one episodic recap to LTM; do not store the full transcript as semantic memory.

Memory, MCP, and the Whole Volume

Independent of framework: working memory is the live transcript/state; semantic LTM is RAG/KB/MCP resources; episodes are recaps; MCP tools are how hosts share actions. AutoGen group chats are infamous for dumping full histories between speakers—summarize or you will burn the context window.

Pick AutoGen when

The UX is a conversation among actors
Human proxy should sit in the same thread
Code/tool pairing with a critic agent
You want speaker policies, not SOP tasks

Avoid / constrain when

You need a drawn, auditable graph
Code execution is on by default
Group chat has no termination cap
Simple RAG or single-tool Q&A suffices

Common Misconception

“AutoGen (or any 15.4 framework) is the agent.” An AI agent is a loop: goal, reason, act, observe, remember, maybe ask a human. Frameworks only package that loop. MCP only cables tools and context. If you cannot draw the loop and the memory stores, switching libraries will not save the design.

Decision Cheat Sheet

If you need…	Start with
Linear doc Q&A	Vol. 14.3 LCEL / LlamaIndex query engine (not an agent)
One actor + a few tools	LangChain agent
Private data + tool branching	LlamaIndex agent
HITL, resume, compliance graph	LangGraph
Named roles and SOPs	CrewAI
Chat/debate + human in-thread	AutoGen
Reusable tools across IDEs/runtimes	MCP servers + any of the above

Knowledge Check

Short Answer: What is AutoGen’s core metaphor? Answer: Conversable agents coordinating via messages / group chat.
True/False: Vol. 14.3 AutoGen already covered RAG indexing in depth. Answer: False—14.3 was orchestration style; 15.4 is agent/HITL/tool loops + capstone choice.
Multiple Choice: human_input_mode="ALWAYS" is: (a) full HITL, (b) CNN pooling, (c) MCP initialize. Answer: (a).
Short Answer: Why disable code_execution_config by default? Answer: Unsandboxed code exec is a high-risk tool loop.
True/False: Group chat transcripts should be dumped wholesale into semantic LTM. Answer: False—compact recaps; transcripts are working memory.
Multiple Choice: Explicit checkpointed HITL graphs: (a) LangGraph, (b) AutoGen only, (c) Volume 05 k-means. Answer: (a).
Short Answer: CrewAI vs AutoGen in one contrast. Answer: Roles/tasks/process vs conversation/speaker policies.
True/False: MCP replaces the need to pick LangGraph or CrewAI. Answer: False—MCP is the tool/resource cable.
Multiple Choice: Single-shot handbook Q&A should usually be: (a) a RAG chain/query engine, (b) a 6-agent group chat, (c) uncensored shell. Answer: (a).
Short Answer: What Volume comes next after this capstone? Answer: Volume 16 (Speech / multimodal I/O).

Key Takeaways

AutoGen = message-passing multi-agent with optional human proxies and tool/code loops.
Cap replies, terminate cleanly, treat code exec as dangerous.
Framework choice: chain vs LC agent vs graph vs index-agent vs crew vs chat.
Memory types + MCP apply to every framework; do not confuse libraries with architecture.
Volume 15 complete. Next: Vol. 16 Speech.

Trainer’s Guide

Capstone lab: Same SSO ticket in (1) LangChain single agent, (2) LangGraph HITL graph, (3) LlamaIndex policy tool, (4) CrewAI three roles, (5) AutoGen assistant+proxy. Students pick one for production and defend with the cheat sheet.

Whiteboard: Full Volume 15 stack: 15.1 loop + 15.2 memory + 15.3 MCP + 15.4 framework. Then preview speech as another I/O channel into working memory.

Recap: AutoGen closes Volume 15 with conversation-centric agents and a framework-choice capstone. Continue with Vol. 16 Speech.