Spend Alerts & Budgets — Module 13.4

How This Lesson Fits the Module & Volume

Spend alerts and budgets give humans time to react before quotas kill traffic. Finance sets a budget; eng wires thresholds, anomaly alerts, and owners on-call.

This is the organizational control loop around per-token COGS.

Related foundations: recount tokens with tiktoken (Vol. 12) and keep payloads inside the context window (Vol. 11).

Learning Objectives

By the end of this lesson, students should be able to:

Define budget vs alert vs hard quota.
Set multi-threshold alerts (50/75/90%).
Detect anomalies vs slow burns.
Route alerts to owners with runbooks.
Include forecast-to-month-end projections.
Avoid alert fatigue with sensible hysteresis.

Definition

Spend alerts and budgets are planned maximum costs plus notifications (and optional automated actions) when actual or forecasted LLM spend crosses thresholds.

Control	Behavior	Example
Budget	Target ceiling	$12k/mo LLM COGS
Alert	Notify	Slack at 75%
Auto-action	Mitigate	Disable non-prod keys at 95%
Quota	Enforce	Hard block at 100%

Code: Threshold Checker

def alert_level(spent: float, budget: float) -> str | None:
    ratio = spent / budget
    if ratio >= 1.0:
        return "critical"
    if ratio >= 0.9:
        return "page"
    if ratio >= 0.75:
        return "warn"
    if ratio >= 0.5:
        return "info"
    return None

def forecast_eom(spent_so_far, day_of_month, days_in_month):
    if day_of_month <= 0:
        return spent_so_far
    return spent_so_far / day_of_month * days_in_month

print(alert_level(9_200, 10_000), forecast_eom(9_200, 20, 30))

Slow burn

Steady climb
Threshold alerts
Trim features/k

Anomaly spike

Sudden jump
Page + circuit break
Kill runaway agent

Forecast breach

Will miss EOM
Act mid-month
Renegotiate budget

Strengths

Humans can intervene early
Ties LLM ops to finance
Supports auto-mitigations

Tradeoffs

Noisy alerts get ignored
Incomplete metering blinds you
Multi-cloud bills lag

Common Misconception

“We only look at the invoice at month end.” By then the agent loop already spent the quarter. Wire mid-period alerts on usage pipelines, not only accounting exports.

Knowledge Check

Short Answer: What is a spend budget here? Answer: A planned maximum LLM cost for a period.
True/False: Alerts should wait for the final invoice only. Answer: False.
Multiple Choice: 75% threshold usually: (a) pages sleep, (b) warns owners, (c) retrains BERT. Answer: (b).
Short Answer: Why forecast end-of-month? Answer: Act before the budget is already blown.
True/False: Anomaly spikes need different playbooks than slow burns. Answer: True.
Multiple Choice: Auto-action example: (a) disable staging keys, (b) delete prod data, (c) ignore meters. Answer: (a).
Short Answer: What causes alert fatigue? Answer: Too many low-value pages without hysteresis/ownership.
Short Answer: What feeds alert pipelines? Answer: Token usage × rates (metering from 13.3/13.4).
Multiple Choice: Budgets without quotas: (a) always enforce, (b) may only notify, (c) set temperature. Answer: (b).
True/False: Every alert needs a named owner and runbook. Answer: True.

Key Takeaways

Budget + multi-threshold alerts + optional auto-action.
Forecast EOM; watch anomalies separately.
Meter continuously, not only on invoices.
Fight alert fatigue with clear owners.
Next: Prompt Caching.

Trainer’s Guide

Lab: Draft a Slack alert template at 75/90/100% with runbook links.

Discussion: When is auto-disabling a feature safer than paging a human?

Recap: Alerts buy time; budgets set intent. Continue with Prompt Caching.