Training Set — Module 5.1

How This Lesson Fits the Module

You have X and y. The training set is the subset of data the algorithm sees during fit—where weights, splits, and patterns are learned. Everything else in the module exists to estimate how well that learning generalizes.

Engineers size and curate training data deliberately: enough volume for signal, clean enough to trust, and never contaminated with test or validation rows.

Learning Objectives

By the end of this lesson, students should be able to:

Define the training set and its role in estimator.fit(X_train, y_train).
Explain what the model learns from training data vs what it does not see.
Recognize underfitting and overfitting in terms of train performance.
Apply basic training-set hygiene: deduplication and class representation.
Connect training size to variance and deployment risk.
Prepare train indices for upcoming split and CV lessons.

What the Training Set Does

The training set is the portion of labeled data used to estimate model parameters. Linear models learn coefficients; trees learn split thresholds; neural nets learn weights. If a pattern is not in training data (or is too noisy), the model cannot learn it reliably.

Phase	Data used	sklearn call
Training	Training set only	`model.fit(X_train, y_train)`
Validation / CV	Held-out folds	`predict` + metric
Final test	Locked test set	One-time evaluation

Fitting a Baseline Model

Start simple: logistic regression or a shallow tree on a solid training slice. Watch training metrics but do not optimize solely on them—high train accuracy with poor validation is overfitting.

from sklearn.linear_model import LogisticRegression

# Assume X_train, y_train already defined by a proper split
clf = LogisticRegression(max_iter=1000, class_weight="balanced")
clf.fit(X_train, y_train)

train_score = clf.score(X_train, y_train)
print(f"Training accuracy: {train_score:.3f}")
# Compare to validation in later lessons — not train alone

Critical Mistake — Training on All Data

Calling fit on the entire dataset including the final test set makes offline metrics meaningless. Reserve test data before any exploratory tuning.

Training Set Size and Quality

More data usually helps generalization, but dirty or duplicated rows inflate train scores without helping production. Balance rows across time periods if the world drifts.

More data helps when

Signal is weak but real
Rare classes need examples
Features are high-dimensional

More data hurts when

Old regime unlike production
Duplicates dominate
Labels are systematically wrong

Engineering Habit — Training Snapshot ID

Log training_snapshot_id, row count, and date range with every model artifact in MLflow or your registry. Retrains should be comparable audits, not mysteries.

Knowledge Check

Short Answer: What happens during fit? Answer: The estimator learns parameters from training features and labels.
True/False: Excellent training accuracy guarantees production success. Answer: False—overfitting and drift exist.
Multiple Choice: Training set is used for: (a) final report to CFO, (b) learning weights, (c) hyperparameter selection only. Answer: (b).
Short Answer: What is overfitting? Answer: Model memorizes training noise and fails on new data.
Short Answer: Why use class_weight="balanced"? Answer: Upweights rare classes during training on imbalanced y.
True/False: Duplicate users in training always help. Answer: False—they bias and inflate metrics.
Multiple Choice: Before fit, you should have: (a) only X, (b) X_train and y_train from a proper split, (c) test labels in X. Answer: (b).
Short Answer: What is underfitting? Answer: Model too simple to capture signal even on training data.
Short Answer: Why log training row count? Answer: Track data volume across retrains for debugging metric shifts.
Multiple Choice: Evaluating on training data alone measures: (a) generalization, (b) memorization fit, (c) production drift. Answer: (b).

Key Takeaways

The training set is where parameters are learned via fit.
High training performance alone is insufficient—validate on held-out data.
Quality, deduplication, and regime relevance matter as much as size.
Next: Testing Set—the locked exam for generalization.

Trainer’s Guide

Hands-on idea: Students train the same model on 10%, 50%, and 100% of eligible rows; plot train vs validation metric to illustrate variance and data hunger.

Discussion prompt: Your training set stops at 2023 but you deploy in 2026. What risk does that create?

Recap: The training set is where the model learns parameters via fit. Continue with Testing Set.