Input Layer — Module 6.1

How This Lesson Fits Module 6.1

Every deep model begins with an input layer that establishes tensor shape, scale, and meaning. Before studying hidden layers, students need a disciplined way to represent examples so downstream weights receive stable numeric signals.

Learning Objectives

By the end of this lesson, students should be able to:

Define the input layer as the model's contract with data tensors.
Identify batch dimension, feature dimension, channel dimension, and sequence length.
Explain why scaling and encoding still matter in neural networks.
Build simple PyTorch inputs for tabular and image-like data.
Detect common tensor-shape mistakes before training.
Connect input design to model architecture choices.

Definition

The input layer is the interface that receives a batch of examples in a fixed tensor shape expected by the first trainable layer.

The Input Layer Is a Shape Contract

A neural network does not understand rows, pixels, words, or measurements until they are converted into tensors. The input layer says what each axis means. For tabular data, the model often expects [batch, features]. For images, it may expect [batch, channels, height, width]. If this contract is wrong, the rest of the model can be mathematically valid but semantically useless.

Data type	Typical tensor shape	First layer
Tabular	`[N, features]`	`nn.Linear(features, hidden)`
Image	`[N, C, H, W]`	Convolution or flatten + linear
Text tokens	`[N, sequence]`	Embedding layer
Time series	`[N, time, features]`	RNN, transformer, or temporal CNN
Single example	Often add batch dimension	`x.unsqueeze(0)`

PyTorch Practice

The first linear layer must match the number of incoming features exactly.

import torch
from torch import nn

batch = 16
features = 12
x = torch.randn(batch, features)

model = nn.Sequential(
    nn.Linear(features, 32),
    nn.ReLU(),
    nn.Linear(32, 1),
)

y_hat = model(x)
print(x.shape)      # [16, 12]
print(y_hat.shape)  # [16, 1]

Input Responsibilities

Shape

Batch axis is usually first
Feature count must match first layer
Images and sequences keep structure

Scale

Normalize continuous values
Avoid huge unit differences
Preserve train-only fitting for scalers

Semantics

Encode categories intentionally
Do not leak labels into features
Document what each column means

Strengths and Tradeoffs

Useful because

A clean input contract makes model bugs easier to diagnose.
Well-scaled inputs improve optimizer behavior.
Correct shapes allow architectures to exploit structure.

Watch for

Silent column order mistakes can train plausible but wrong models.
Flattening structured data too early can discard useful locality.
Preprocessing fit on all data can leak test information.

How It Flows

1. Collect

Gather raw examples and targets with stable identifiers.

2. Encode

Convert text, categories, and measurements into numbers.

3. Scale

Fit transformations on training data and apply consistently.

4. Batch

Stack examples into tensors with the expected axes.

5. Validate

Print shapes and sample values before training.

Common Misconception

The most common input-layer bug is a model that runs but receives the wrong columns, wrong order, or leaked target-derived features. Shape correctness is necessary, but it is not enough.

Knowledge Check

Short Answer: What does the input layer define? Answer: The tensor shape and data contract entering the model.
True/False: Batch dimension is commonly the first axis in PyTorch. Answer: True.
Multiple Choice: Tabular input usually has shape: (a) [N, features], (b) [H, W], (c) [classes]. Answer: (a).
Short Answer: Why normalize continuous inputs? Answer: To improve numerical stability and optimizer behavior.
True/False: Input preprocessing can cause data leakage. Answer: True.
Short Answer: Which PyTorch layer often follows token IDs? Answer: An embedding layer.
Multiple Choice: Image tensors in PyTorch are often: (a) [N,C,H,W], (b) [N,H,W,C], (c) [classes,N]. Answer: (a).
Short Answer: What must match an nn.Linear first argument? Answer: The incoming feature dimension.
True/False: A single example may need unsqueeze(0) to add a batch dimension. Answer: True.
Short Answer: Why document column meaning? Answer: To prevent semantic drift and wrong-feature training.

Key Takeaways

The input layer is the model's data contract.
Shape, scale, and semantics must all be correct.
Good input discipline prevents many expensive training failures.
Next, Hidden Layer shows how the model transforms inputs into representations.

Trainer’s Guide

Hands-on idea: Give students three tensors and ask them to identify which architecture each tensor shape suggests before writing any model code.

Discussion prompt: Which is more dangerous in production: a visible shape error or an invisible column-order error? Why?

Recap: The input layer defines how real data becomes tensors the network can meaningfully process. Continue with Hidden Layer.