Input vs Output Pricing Tiers — Module 13.4

How This Lesson Fits the Module & Volume

Price cards split input vs output tiers—and sometimes cached, batch, or premium reasoning tiers. Optimizing the wrong side wastes effort.

Extends 13.3’s input/output token split into explicit dollar tiers.

Related foundations: recount tokens with tiktoken (Vol. 12) and keep payloads inside the context window (Vol. 11).

Learning Objectives

By the end of this lesson, students should be able to:

Read asymmetric $/1M input vs output rates.
Identify which workloads are input-heavy vs output-heavy.
Prioritize optimizations on the expensive side.
Account for extra tiers (cache hit, batch, flex).
Model break-evens when shortening outputs vs retrieval.
Avoid averaging rates into one misleading number.

Definition

Input vs output pricing tiers are distinct per-tokens rates (and SKUs) applied to prompt tokens versus completion tokens—often with further tiers for cache hits or async batch.

Workload	Dominant side	First lever
RAG Q&A	Input	Chunking, top-k, caching
Long-form generation	Output	max_tokens, outlines
Tool calling	Mixed	Schema size + short args
Classification	Input (short out)	Small model routing

Code: Side-Aware Estimate

def estimate(prompt_t, out_t, in_per_m, out_per_m, cache_hit_t=0, cache_per_m=None):
    cache_per_m = cache_per_m if cache_per_m is not None else in_per_m
    billable_in = max(prompt_t - cache_hit_t, 0)
    return (
        billable_in / 1e6 * in_per_m
        + cache_hit_t / 1e6 * cache_per_m
        + out_t / 1e6 * out_per_m
    )

# Example: output 4x input price
print(estimate(10_000, 500, 2.0, 8.0))
print(estimate(10_000, 500, 2.0, 8.0, cache_hit_t=8_000, cache_per_m=0.5))

Input tier

Prefill / prompt
RAG sensitive
Caching helps

Output tier

Decode / completion
CoT sensitive
Caps help

Special tiers

Cache/batch/flex
Big discounts
Constraints apply

Strengths

Targets spend where it hurts
Explains why concise answers save $
Supports cache ROI math

Tradeoffs

More columns to maintain
Avg blended rate misleads finance
Hidden reasoning tokens on some SKUs

Common Misconception

“I’ll use the average of input and output prices for all planning.” Blended averages hide that a chatty agent can burn the output tier while RAG burns input.

Knowledge Check

Short Answer: Why split input vs output prices? Answer: Different costs/margins; workloads stress sides differently.
True/False: RAG is usually input-heavy. Answer: True.
Multiple Choice: Long Chain-of-Thought mainly hits: (a) output tier, (b) JPEG tier, (c) DNS tier. Answer: (a).
Short Answer: Name a special discounted input tier. Answer: Cached prompt tokens (or batch).
True/False: A single blended $/token is enough for agent vs RAG planning. Answer: False.
Multiple Choice: max_tokens primarily bounds spend on: (a) output tier, (b) image DPI, (c) CSS. Answer: (a).
Short Answer: First lever for input-heavy apps? Answer: Retrieval/chunking/caching (not endless max_tokens).
Short Answer: How does 13.3 help? Answer: Separates input vs output token counts for the formula.
Multiple Choice: Tool JSON schemas mostly add: (a) input tokens, (b) GPU fans, (c) CSS rem. Answer: (a).
True/False: Output is often priced higher per token than input. Answer: True.

Key Takeaways

Never blend away the input/output split.
Optimize the dominant expensive side.
Include cache/batch tiers in the matrix.
Tie levers to workload shape.
Next: Cost Estimation.

Trainer’s Guide

Lab: Classify 10 production traces as input- vs output-heavy; pick one lever each.

Discussion: When would you accept higher output rates for better UX?

Recap: Tiers tell you which side of the meter to squeeze. Continue with Cost Estimation.