Decoder Only — Module 11.3

How This Lesson Fits the Module & Volume

The GPT lectures showed the product lineage; now we name the architecture pattern they share. Decoder-only Transformers (Volume 10 decoder stack + causal mask) power GPT-style LMs—contrasting Module 11.2’s encoder-only BERT family and classic encoder–decoder seq2seq.

Learning Objectives

By the end of this lesson, students should be able to:

Define decoder-only LMs and their causal self-attention constraint.
Compare encoder-only, decoder-only, and encoder–decoder layouts.
Trace embeddings → stacked decoder blocks → LM head.
Implement a tiny causal decoder block sketch in PyTorch.
Explain why one stack suffices for both understanding and generation in LLMs.
Link to autoregressive factorization and causal attention.

Definition

A decoder-only language model is a Transformer that stacks decoder-style blocks with masked (causal) self-attention so each position may attend only to itself and prior positions, trained to predict the next token—no separate bidirectional encoder stack required.

Three Layouts

Encoder-only

BERT family
Bidirectional
NLU heads

Decoder-only

GPT family
Causal mask
Generation native

Encoder–Decoder

Classic NMT / T5
Cross-attention
Seq2seq tasks

Embed

Tokens + positions.

Decode stack

Causal MHA + FFN × N.

LM head

Logits over vocabulary.

Sample

Next token → append.

import torch
from torch import nn

class CausalDecoderBlock(nn.Module):
    def __init__(self, d=64, nhead=4):
        super().__init__()
        self.attn = nn.MultiheadAttention(d, nhead, batch_first=True)
        self.ff = nn.Sequential(nn.Linear(d, 4*d), nn.GELU(), nn.Linear(4*d, d))
        self.n1, self.n2 = nn.LayerNorm(d), nn.LayerNorm(d)

    def forward(self, x):
        T = x.size(1)
        causal = torch.triu(torch.ones(T, T, dtype=torch.bool, device=x.device), 1)
        h = self.n1(x)
        a, _ = self.attn(h, h, h, attn_mask=causal)
        x = x + a
        x = x + self.ff(self.n2(x))
        return x

blk = CausalDecoderBlock()
print(blk(torch.randn(2, 8, 64)).shape)

Common Misconception

“Decoder-only models cannot understand text; they only babble.” They build rich contextual states—optimized for left-to-right prediction—and underpin modern assistants. Understanding vs. generation is more about objective and interface than magic encoder molecules.

Strengths and Tradeoffs

Strengths

One stack for pretrain and generation.
Scales into LLMs cleanly.
Natural streaming token output.

Tradeoffs

No native future context.
Quadratic attention cost with length.
Bidirectional tasks may prefer encoders.

Knowledge Check

Short Answer: What mask defines decoder-only LMs? Answer: Causal (look-ahead) mask.
True/False: Decoder-only GPT stacks include a separate BERT encoder by default. Answer: False.
Multiple Choice: BERT is: (a) encoder-only, (b) decoder-only, (c) a decision tree. Answer: (a).
Short Answer: What sits atop the stack for next-token prediction? Answer: An LM head (linear to vocab).
True/False: During generation we append sampled tokens to the context. Answer: True.
Multiple Choice: Cross-attention to an encoder output is central in: (a) pure GPT decoder-only, (b) encoder–decoder Transformers, (c) k-means. Answer: (b).
Short Answer: Name one reason LLMs favor decoder-only stacks. Answer: Unified generative pretraining and serving simplicity.
Short Answer: Which lecture details the mask math? Answer: Causal Attention.
Multiple Choice: Position information is: (a) still required, (b) never used, (c) only for CNNs. Answer: (a).
True/False: Decoder-only implies the model cannot be instruction-tuned. Answer: False.

Key Takeaways

GPT-style models are decoder-only Transformers with causal attention.
Contrast with BERT encoders and seq2seq encoder–decoders.
Pipeline: embed → causal blocks → LM head → sample.
One stack scales to modern LLMs.
Next: Autoregressive Model.

Trainer’s Guide

Hands-on idea: Print a T×T causal mask matrix for T=6 and verify upper triangle is blocked.

Discussion prompt: When would you still pick an encoder–decoder over decoder-only?

Recap: Decoder-only is the GPT architectural home. Continue with Autoregressive Model.