Input Tokens vs Output Tokens — Module 13.3

How This Lesson Fits the Module & Volume

Providers price and limit input (prompt) and output (completion) tokens differently. Mixing them up breaks budgets, max_tokens settings, and Module 13.4 cost math.

Counts still come from the same tokenizer family (tiktoken), but the billable buckets and control knobs diverge.

Learning Objectives

By the end of this lesson, students should be able to:

Define input vs output tokens in API usage objects.
Explain why output is often priced higher per token.
Set completion limits without starving the prompt.
Estimate asymmetric cost: large prompt, short answer (and vice versa).
Account for reasoning/hidden tokens when a provider exposes them.
Design prompts that shift work to cheaper input when appropriate.

Definition

Input tokens are tokens the model consumes as context (system, user, tools, history). Output tokens are tokens the model generates. Both count toward the context window, but they are metered and priced as separate lines.

Aspect	Input	Output
Also called	Prompt tokens	Completion tokens
Control	Your packing/truncation	`max_tokens` / stop
Typical price	Lower per 1M	Higher per 1M
Latency link	Prefill cost	Decode cost (sequential)

Code: Read Both From Usage

# Pseudocode against a Chat Completions-style response
usage = response.usage
prompt_tokens = usage.prompt_tokens      # input
completion_tokens = usage.completion_tokens  # output
total = usage.total_tokens

# Cost sketch (prices are examples — look up current rates):
IN_PER_M, OUT_PER_M = 2.50, 10.00
cost = prompt_tokens / 1e6 * IN_PER_M + completion_tokens / 1e6 * OUT_PER_M
print(prompt_tokens, completion_tokens, f"${cost:.6f}")

Input-heavy

RAG dumps, long PDFs
Optimize retrieval/chunking
Cache stable prefixes

Output-heavy

Long essays, CoT
Cap max_tokens
Ask for concise formats

Balanced

Chat with moderate replies
Watch both meters
Budget each side

Strengths

Clear billing lines
Lets you optimize each side
Maps to prefill vs decode

Tradeoffs

Students confuse total with either side
Some APIs add extra usage fields
Streaming needs careful aggregation

Common Misconception

“max_tokens limits the whole request size.” It caps generated tokens (with provider-specific nuances). Prompt size is controlled by what you send, not by max_tokens alone.

Knowledge Check

Short Answer: What are input tokens? Answer: Tokens in the prompt/context the model reads.
Short Answer: What are output tokens? Answer: Tokens the model generates as the completion.
True/False: Output tokens are often priced higher than input. Answer: True.
Multiple Choice: max_tokens primarily caps: (a) PDF pages, (b) generated tokens, (c) embedding dims. Answer: (b).
True/False: Input and output both consume the context window. Answer: True.
Multiple Choice: RAG document packing mainly grows: (a) output, (b) input, (c) temperature. Answer: (b).
Short Answer: Why care about the split for cost? Answer: Different $/token rates make totals sensitive to which side dominates.
Short Answer: Which Vol. 12 library counts either side locally before calling? Answer: tiktoken (for text encodings).
Multiple Choice: Long Chain-of-Thought mainly inflates: (a) output tokens, (b) image size, (c) batch size only. Answer: (a).
True/False: total_tokens alone is enough for tiered pricing analysis. Answer: False—split input/output.

Key Takeaways

Meter input and output separately.
Price and controls differ by side.
max_tokens is not a total-request cap.
Optimize the expensive side first.
Next: Token Limits.

Trainer’s Guide

Lab: Take one RAG and one long-form generation workload; estimate cost with asymmetric rates.

Discussion: When should you move reasoning into the prompt (examples) vs asking the model to think aloud?

Recap: Input vs output is the accounting split behind limits and bills. Continue with Token Limits.