Feature — Module 5.1

How This Lesson Fits the Module

You defined the dataset as rows of examples. Features are the input columns the model uses to make predictions—the signals in X. Volume 04 feature engineering built many of them; here you decide which belong in the training matrix and how sklearn will consume them.

Bad feature choices cause more production failures than wrong algorithms. AI engineers treat features as versioned, documented inputs with the same definitions offline and online.

Learning Objectives

By the end of this lesson, students should be able to:

Define features (X) and distinguish them from labels and metadata.
Classify features as numeric, categorical, ordinal, or derived.
Build a feature matrix with pandas and pass it to sklearn estimators.
Apply ColumnTransformer for mixed-type preprocessing.
Reject leaky or non-servable columns before training.
Align feature lists between training exports and serving APIs.

Features as Model Inputs

A feature is any column (or derived vector) that encodes information available at prediction time. The model learns weights or split rules that map features to outputs. Everything not in X is invisible to the learner—including brilliant columns you forgot to include.

Feature type	Examples	Typical preprocessing
Numeric continuous	`tenure_days`, `avg_order_value`	Scaling, log transform
Numeric discrete	`support_tickets_30d`	Sometimes treat as numeric or bucket
Nominal categorical	`country`, `plan_tier`	One-hot encoding
Ordinal categorical	`education_level`	Ordinal encoding with true order
Boolean	`is_mobile_user`	Cast to 0/1

Building X with pandas

Select features explicitly—never pass the whole dataframe and hope. Drop IDs, free-text blobs (unless embedded elsewhere), and post-outcome columns flagged in Volume 04.

import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler

num_cols = ["tenure_days", "orders_last_90d", "avg_order_value"]
cat_cols = ["plan_tier", "country"]

X = df[num_cols + cat_cols].copy()
X["avg_order_value"] = X["avg_order_value"].fillna(0)

preprocess = ColumnTransformer(
    transformers=[
        ("num", StandardScaler(), num_cols),
        ("cat", OneHotEncoder(handle_unknown="ignore"), cat_cols),
    ]
)

# X is still raw here — preprocess.fit_transform runs inside Pipeline later

Critical Mistake — Including the Label in X

Accidentally leaving churned_30d or a proxy like refund_issued in X produces fake accuracy. Automate column lists in config files; never rely on “drop everything except last column.”

Feature Lists and Train/Serve Parity

Production APIs must accept the same features in the same order (or named columns with a fixed schema). Drift between offline X and online payloads is a top cause of silent model degradation.

Practice	Why it matters
Versioned `feature_config.yaml`	Single source of truth for train and serve
Schema validation on ingest	Reject bad requests before inference
Default values documented	Missing features handled consistently
Point-in-time joins in SQL	Features match prediction moment

Engineering Habit — Feature Smoke Test

Before training, assert set(train_columns) == set(serving_columns) and run one live request through the same transform path as the notebook. Mismatches should fail CI.

Knowledge Check

Short Answer: What symbol usually denotes features in sklearn code? Answer: X (feature matrix).
True/False: user_id should routinely be a numeric feature. Answer: False—use for grouping, not as a learned signal.
Multiple Choice: Best encoding for low-cardinality country: (a) raw strings in XGBoost only, (b) one-hot, (c) row index. Answer: (b) for linear models; trees may differ.
Short Answer: What does ColumnTransformer do? Answer: Applies different preprocessors to different column subsets.
Short Answer: Why document default fill for missing features? Answer: Train and serve must impute identically.
True/False: More features always improve generalization. Answer: False—noise and leakage can hurt.
Multiple Choice: Leaky column to exclude: (a) tenure_days, (b) cancellation_email_sent, (c) plan_tier. Answer: (b).
Short Answer: Nominal vs ordinal categorical? Answer: Nominal has no order; ordinal has meaningful order.
Short Answer: What is train/serve skew? Answer: Offline features differ from production feature computation.
Multiple Choice: handle_unknown="ignore" helps when: (a) test set has unseen categories, (b) labels are missing, (c) GPU is slow. Answer: (a).

Key Takeaways

Features are prediction-time inputs in X—curated, typed, and documented.
Mixed-type tabular data needs column-aware preprocessing.
Exclude labels, IDs, and leaky proxies from the feature matrix.
Next: Label—the target y the model learns to predict.

Trainer’s Guide

Hands-on idea: From a shared churn dataframe, pairs defend their num_cols / cat_cols lists and present one column they rejected with a leakage or serve-time argument.

Discussion prompt: A PM asks to add “customer lifetime value to date.” Is it always valid at the prediction moment?

Recap: Features are the input columns the model sees; choose ones available at prediction time. Continue with Label.