Query — Module 10.1

How This Lesson Fits the Module & Volume

You now know the encoder writes memory and the decoder generates while reading that memory. Attention itself needs a vocabulary for the vectors that participate in the match: Query, Key, and Value.

This lecture isolates the query—the vector that asks “what am I looking for right now?” In decoder cross-attention, queries come from the current decoder state; in self-attention, each position forms its own query from its hidden state. Mastering Q first makes the rest of Module 10.1 read like a retrieval system.

Learning Objectives

By the end of this lesson, students should be able to:

Define the query as the attention request vector derived from a hidden state.
Explain how a learned projection W_Q maps hidden states to query space.
Contrast queries in self-attention vs. cross-attention.
Relate queries to similarity search: Q is compared against keys.
Implement query projection in PyTorch and verify tensor shapes.
Preview how queries drive attention scores.

Definition

A query is a vector q = x W_Q (or a batch of such vectors) that represents what a position is searching for. Attention compares each query to a set of keys; the resulting weights mix values.

Retrieval Metaphor

Think of a search engine: your search string is the query; document titles (or embeddings) are keys; document contents are values. Attention is soft retrieval over a finite set of positions instead of a web index. The query does not copy content—it only decides whom to read.

Role	Analogy	In the model
Query (Q)	Search request	Projected from the “reader” state
Key (K)	Index / address	Projected from candidate states
Value (V)	Payload content	Projected content that gets mixed

Where Queries Come From

Self-Attention

Same sequence produces Q, K, and V.
Each token asks about all tokens (or past ones if masked).
Encoder: full; decoder: causal.

Cross-Attention

Q from decoder hidden states.
K, V from encoder memory.
Lets generation look at the source.

Learned Projection

W_Q is a linear layer (no bias often).
Maps d_model → d_k (per head).
Shared across positions, not across roles.

Query Projection in PyTorch

import torch
from torch import nn

B, T, d_model, d_k = 2, 5, 64, 16
x = torch.randn(B, T, d_model)          # hidden states (e.g. decoder)

W_Q = nn.Linear(d_model, d_k, bias=False)
Q = W_Q(x)                              # (B, T, d_k)

print(Q.shape)                          # torch.Size([2, 5, 16])
# Each of the T positions now has a query vector in R^d_k

Queries and Scores

Given keys K of shape (B, S, d_k), the unnormalized attention scores are typically Q K^T → (B, T, S): for every query position, a score against every key position. Scaling and softmax come later in Scaled Dot-Product Attention.

Why a Separate Q Space?

Lets “asking” differ from “being found.”
Learnable geometry for matching.
Enables multi-head specialization.

Pitfalls

Confusing Q with the raw embedding.
Mismatched d_k between Q and K.
Forgetting causal masks on decoder queries.

Common Misconception

“The query is the word we attend to.” Opposite: the query is the word (or decoder step) that is doing the looking. The positions being looked up are represented by keys; the content mixed in comes from values.

Knowledge Check

Short Answer: In one phrase, what does a query represent? Answer: What the current position is looking for / searching for.
True/False: Queries are usually raw embeddings with no projection. Answer: False—they are typically x W_Q.
Multiple Choice: In cross-attention, queries come from: (a) the encoder, (b) the decoder, (c) the vocabulary. Answer: (b).
Short Answer: If Q is (B, T, d_k) and K is (B, S, d_k), what is the shape of Q K^T? Answer: (B, T, S).
True/False: In self-attention, Q, K, and V can all come from the same sequence. Answer: True.
Multiple Choice: W_Q maps: (a) d_k → vocab, (b) d_model → d_k, (c) T → S. Answer: (b).
Short Answer: Name the retrieval analogy for the query. Answer: The search request / search string.
True/False: Softmax is applied to queries alone before seeing keys. Answer: False—softmax is over scores between queries and keys.
Multiple Choice: Decoder self-attention queries must respect: (a) no mask, (b) a causal mask, (c) only PAD masking. Answer: (b).
Short Answer: Which lecture covers comparing Q to K? Answer: Attention Score (and Scaled Dot-Product Attention).

Key Takeaways

The query is the “search request” vector: q = x W_Q.
Self-attention and cross-attention differ mainly in where Q vs. K/V come from.
Scores arise from comparing queries to keys; values are mixed afterward.
Next: Key—the address side of the match.

Trainer’s Guide

Hands-on idea: Have students print Q for two different sentences that share a word and discuss how context changes the query.

Discussion prompt: Why learn three matrices W_Q, W_K, W_V instead of reusing the hidden state as all three?

Recap: Queries ask; keys answer the match; values supply content. Continue with Key.