Chroma / ChromaDB — Module 14.2

How This Lesson Fits the Module & Volume

After FAISS, most teams want documents, metadata, and embeddings in one API without wiring a side store. Chroma (ChromaDB) is the developer-first open-source vector database popular in RAG tutorials and LangChain demos.

It sits between “library” and “full cluster”: local persistence by default, optional client/server, first-class metadata filters—ideal for learning production-shaped RAG before Pinecone/Milvus scale.

Learning Objectives

By the end of this lesson, students should be able to:

Create persistent Chroma collections and add documents with embeddings/metadata.
Query with where metadata filters and interpret results.
Contrast ephemeral in-memory vs persistent local vs client/server modes.
Explain when Chroma is enough for MVPs and when to migrate.
Connect Chroma to the RAG pipeline from Module 14.1 (chunks + embeddings).
Compare Chroma’s DX focus to FAISS (raw ANN) and Pinecone (managed SaaS).

Definition

Chroma is an open-source embedding database that stores documents, embeddings, and metadata in collections, and exposes simple APIs for add, query, update, and delete—optimized for AI application developers rather than cluster operators.

Where Chroma Fits

Mode	Use when	Watch for
In-memory	Unit tests, throwaway notebooks	Data gone on exit
Persistent local	Laptops, single-box RAG apps	Single-node limits
Client / server	Shared team services	Ops still lighter than Milvus
Cloud / hosted	Managed path without self-host	Vendor coupling, cost

Collections, Documents, Metadata

A collection is a named bucket of vectors plus optional documents and metadata dicts. Filters use those metadata fields—the gap you felt with FAISS.

import chromadb
from chromadb.utils import embedding_functions

client = chromadb.PersistentClient(path="./chroma_store")
embed_fn = embedding_functions.DefaultEmbeddingFunction()

col = client.get_or_create_collection(
    name="support_docs",
    embedding_function=embed_fn,
    metadata={"hnsw:space": "cosine"},
)

col.add(
    ids=["c1", "c2", "c3"],
    documents=[
        "Reset password via Settings > Security.",
        "Enterprise SSO uses SAML with Okta.",
        "Billing invoices appear under Workspace > Billing.",
    ],
    metadatas=[
        {"product": "auth", "tier": "all"},
        {"product": "auth", "tier": "enterprise"},
        {"product": "billing", "tier": "all"},
    ],
)

hits = col.query(
    query_texts=["How do I turn on SSO?"],
    n_results=2,
    where={"product": "auth"},           # metadata filter
)
print(hits["documents"][0])
print(hits["metadatas"][0])
print(hits["distances"][0])

Filtering Patterns

Embed query

Text → vector

ANN search

Top candidates

Metadata where

tenant, tags, dates

Return chunks

Into RAG prompt

Prefer storing stable keys (source, doc_id, section, acl) so filters mirror product rules. Over-filtering empty results is a common RAG bug—log filter hit rates.

Strengths

Minutes to first RAG demo
Documents + metadata + vectors together
Great LangChain / LlamaIndex adapters
Local-first for privacy prototypes

Tradeoffs

Not the ceiling for billion-scale HA
Index/ops knobs fewer than Milvus
Team must still design chunk schemas
Migration planning if you outgrow it

Common Misconception

“Chroma auto-solves retrieval quality.” It stores and searches; quality still depends on chunking, embedding choice, filters, and reranking. A wrong where clause can zero out relevant hits.

Knowledge Check

Short Answer: What three things does a Chroma collection typically store? Answer: Embeddings, documents (text), and metadata.
True/False: PersistentClient keeps data on disk across restarts. Answer: True.
Multiple Choice: Metadata filters are passed via: (a) where, (b) nprobe, (c) tokenizer. Answer: (a).
Short Answer: Name one Chroma mode besides persistent local. Answer: In-memory, client/server, or hosted/cloud.
True/False: Chroma replaces the need for good chunking. Answer: False.
Multiple Choice: Compared to FAISS, Chroma emphasizes: (a) GPU kernels only, (b) developer DX + metadata, (c) SQL OLAP. Answer: (b).
Short Answer: Why log empty-filter rates? Answer: Over-strict where clauses can hide relevant documents.
Short Answer: When might you migrate off Chroma? Answer: Multi-region HA, extreme scale, stricter SLAs, etc.
Multiple Choice: get_or_create_collection is useful to: (a) train LLMs, (b) idempotently open a named store, (c) tokenize PDF. Answer: (b).
True/False: Chroma is commonly used in LangChain RAG tutorials. Answer: True.

Key Takeaways

Chroma packages vectors, text, and metadata for fast local RAG development.
Use where filters to enforce product, tenant, and ACL constraints.
Choose persistence mode to match tests vs demos vs shared services.
Plan a growth path to Pinecone/Milvus/Qdrant if scale or ops demand it.
Next: Pinecone—fully managed vector search as a service.

Trainer’s Guide

Lab: Ingest 20 support snippets with product/tier metadata; query with and without where and compare hit sets.

Debrief: Draw the line between “Chroma MVP” and “time to managed/self-hosted cluster.”

Recap: Chroma is the friendly local vector DB for RAG prototypes. Continue with Pinecone.