Set

Module 3.1 Capstone: Unique Elements in AI Systems

You have learned variables, types, operators, loops, functions, lists, dictionaries, and tuples. Sets complete the core collection types: unordered containers that hold unique elements only. Vocabulary tokens, class labels, user IDs in a cohort, and feature names in a schema—all are set problems in disguise.

Sets power deduplication in data cleaning, fast membership tests for stopwords, and classical NLP metrics (precision/recall via intersection and union). This lecture ties Module 3.1 together before you advance to Module 3.2: Object-Oriented Programming.

Learning Objectives

By the end of this lesson, students should be able to:

Create sets with literals, set(), and set comprehensions.
Perform union, intersection, difference, and symmetric difference.
Test membership with in in O(1) average time.
Deduplicate records and extract unique labels from datasets.
Compare sets for vocabulary overlap and class distribution checks.
Choose the right collection type across Module 3.1 types.

Creating Sets

Definition — Set

A set is an unordered collection of unique, hashable elements. Duplicates are silently removed. Sets support fast membership testing and mathematical set operations—ideal for vocabularies and unique ID tracking.

# Unique class labels in a classification dataset
raw_labels = [0, 1, 1, 2, 0, 2, 2, 1, 0]
unique_labels = set(raw_labels)
print(unique_labels)          # {0, 1, 2}
print(len(unique_labels))   # 3 classes

# Vocabulary from tokenized text (deduplicated)
tokens = ["the", "cat", "sat", "the", "mat", "cat"]
vocabulary = set(tokens)
print(vocabulary)             # unordered unique tokens

# Empty set — must use set(), not {}
empty = set()

Set Operations

Operation	Syntax	ML Meaning
Union	`A \| B` or `A.union(B)`	All tokens in either vocabulary
Intersection	`A & B`	Shared features or overlapping vocab
Difference	`A - B`	Tokens in A but not in B (OOV candidates)
Symmetric diff	`A ^ B`	Elements in exactly one set
Membership	`x in A`	Stopword check, label validation

train_vocab = set(["ai", "model", "data", "train"])
val_vocab = set(["ai", "model", "val", "test"])

# Tokens in validation but not training — potential OOV
oov_tokens = val_vocab - train_vocab
print(oov_tokens)   # {'val', 'test'}

# Overlap — shared vocabulary
shared = train_vocab & val_vocab
print(shared)       # {'ai', 'model'}

# Jaccard similarity — overlap measure
jaccard = len(shared) / len(train_vocab | val_vocab)
print(f"Jaccard={jaccard:.2f}")

Deduplication and Data Quality

# Duplicate user IDs in event log — find uniques
user_ids = [101, 202, 101, 303, 202, 404]
unique_users = set(user_ids)
print(f"{len(user_ids)} events, {len(unique_users)} unique users")

# Stopword filter using set membership — O(1) per token
stopwords = {"the", "a", "an", "is", "in"}
words = ["the", "model", "is", "in", "production"]
filtered = [w for w in words if w not in stopwords]
print(filtered)   # ['model', 'production']

Module 3.1 Capstone: Choosing the Right Collection

List

Ordered, duplicates OK
Epoch losses, batches
Index/slice access

Dictionary

Key → value mapping
Configs, metrics, JSON
Named fields

Tuple

Ordered, immutable
Shapes, stat pairs
Hashable records

Unordered, unique only
Vocab, labels, IDs
Fast in checks

Variables — Names for values ↓ Data Types & Operators — Scalars and expressions ↓ Loops & Functions — Repetition and reuse ↓ Lists, Dicts, Tuples — Core collections ↓ Set — Unique elements and set algebra

Python Basics Tool	Volume 02 Math Link	Upcoming in Vol. 03
Variables & floats	Scalars, learning rate, loss	NumPy arrays (Module 3.3)
Lists & loops	Vectors, iterative gradient descent	PyTorch tensors, DataLoader
Functions	Functions f(x), loss ℒ	Classes, methods (Module 3.2)
Dicts & sets	PMFs, vocabularies, event sets	JSON configs, sklearn pipelines

Bridge to Module 3.2With Python basics complete, Module 3.2: Object-Oriented Programming introduces classes—the pattern behind PyTorch nn.Module, custom datasets, and production ML services.

Common Misconceptions

Misconception 1: “Sets preserve insertion order for modeling.”

Why people believe it: Python 3.7+ dicts are ordered, so sets might seem ordered too.

Reality: Sets are unordered. Do not index my_set[0]. Convert to sorted list if you need deterministic iteration.

Misconception 2: “{} creates an empty set.”

Why people believe it: Curly braces appear in set literals like {1, 2}.

Reality: {} is an empty dictionary. Use set() for an empty set.

Misconception 3: “Sets replace lists for all collections.”

Why people believe it: Deduplication is convenient.

Reality: Sets discard order and duplicates by design. Training logs and sequences need lists; unique vocabularies need sets.

Quick Knowledge Check

Short Answer: How to get unique values from a list? Answer: set(my_list) or manual dedup.
True/False: Sets allow duplicate elements. Answer: False.
Multiple Choice: A & B is: (a) union, (b) intersection, (c) difference, (d) product. Answer: (b).
Short Answer: Syntax for empty set? Answer: set().
True/False: x in stopwords is fast for large stopword sets. Answer: True — average O(1).
Computation: {1,2,3} - {2,3,4} equals? Answer: {1}.
Short Answer: What is Jaccard similarity? Answer: |A ∩ B| / |A ∪ B| — overlap fraction.
True/False: Lists are hashable and can be set elements. Answer: False.
Multiple Choice: Best structure for unique class labels: (a) list, (b) set, (c) dict, (d) str. Answer: (b).
Short Answer: What Module 3.2 topic builds on functions with class? Answer: Object-oriented programming (classes, objects).

Key Takeaways

Sets store unique, unordered, hashable elements with fast membership tests.
Union, intersection, and difference support vocabulary overlap and OOV analysis.
Deduplicate IDs and extract unique labels during data exploration.
Module 3.1 completes Python’s core types: variables, scalars, control flow, functions, and four collections.
Volume 02 math is now expressible in Python; Module 3.2 adds classes for models and datasets.
Continue to Module 3.2: Object-Oriented Programming.

Trainer’s Guide

Capstone activity: Given train and validation token lists, students compute unique vocabularies, OOV tokens, and Jaccard similarity—using only Module 3.1 tools.

Collection quiz: Present five AI scenarios; students vote list vs dict vs tuple vs set and justify.

Bridge discussion: Preview how a PyTorch Dataset class will wrap the list-of-samples pattern from this module.

Expected difficulty: Students confuse {} dict vs set(). Drill empty collection literals.

What’s Next You have completed Module 3.1: Python Basics. Continue to Module 3.2: Object-Oriented Programming to build classes, objects, and the patterns underlying PyTorch models and custom datasets.