F1 Score — Module 19.1

How This Lesson Fits the Module & Volume

You now have precision and recall. Stakeholders still want one number to put on a Vol. 18 dashboard. F1 is that compromise: the harmonic mean of P and R. Vol. 05 used it for imbalanced classification; Vol. 19 uses it as a ship/no-ship gate when both FPs and FNs hurt, then shows why Fβ and per-class F1 are often more honest.

After F1, ROC drops the single threshold. In NLP, ROUGE-F and BLEU’s blend of n-gram precision + brevity are cousins of the same idea—combine coverage and cleanliness.

Learning Objectives

By the end of this lesson, students should be able to:

Write F1 as the harmonic mean of precision and recall and explain why harmonic, not arithmetic.
Compute binary, micro, macro, and weighted F1 in sklearn.
Use Fβ (F2 / F0.5) when recall or precision should weigh more.
Choose F1 vs accuracy vs ROC-AUC for a shipped classifier.
Read F1 from classification_report without ignoring support.
See the same P/R tension later in BLEU vs ROUGE-F.

Definition

The F1 score is the harmonic mean of precision and recall: F1 = 2 · P · R / (P + R), equivalently 2 TP / (2 TP + FP + FN). Harmonic mean is dominated by the smaller of P and R: if either is near zero, F1 is near zero. True negatives never enter F1. Fβ generalizes: Fβ = (1 + β²) · P · R / (β² · P + R); β>1 emphasizes recall (F2 is common for screening).

Why Harmonic, Not Average

Arithmetic mean of P=1.0 and R=0.01 is ~0.50—looks “half good.” Harmonic mean is ~0.02. F1 refuses to let a one-sided model hide. That is exactly the dummy always-positive (R=1, P=prevalence) or always-negative (R=0) problem from the accuracy lecture.

P	R	Arithmetic mean	F1
0.90	0.90	0.90	0.90
0.99	0.10	0.545	0.182
0.10	0.99	0.545	0.182
0.50	0.50	0.50	0.50

sklearn: F1, Fβ, and Averages

from sklearn.metrics import (
    f1_score, fbeta_score, classification_report, precision_recall_fscore_support
)
import numpy as np

y_true = np.array([1, 1, 1, 0, 0, 0, 0, 0])
y_pred = np.array([1, 1, 0, 1, 0, 0, 0, 0])
# TP=2, FP=1, FN=1 → P=2/3, R=2/3, F1=2/3

print("f1", f1_score(y_true, y_pred))
print("f2 recall-heavy", fbeta_score(y_true, y_pred, beta=2))
print("f0.5 precision-heavy", fbeta_score(y_true, y_pred, beta=0.5))
print(classification_report(y_true, y_pred, digits=3))

# Multiclass intent router (Vol. 18 API): never report a lone micro-F1
y_true_m = np.array([0, 0, 0, 0, 1, 1, 2, 2, 2])
y_pred_m = np.array([0, 0, 0, 1, 1, 1, 2, 0, 2])
for avg in ("micro", "macro", "weighted"):
    print(avg, f1_score(y_true_m, y_pred_m, average=avg))
print("per-class", f1_score(y_true_m, y_pred_m, average=None))

Which Average to Put on the Dashboard

Binary F1

One positive class that matters
Fraud, churn, toxicity flag
Still show P and R beside it

Macro F1

Every intent/SKU is a promise
Rare classes equal weight
Default for multiclass product APIs

Micro / weighted

Micro ≈ accuracy (single-label)
Weighted follows support
Can hide a dead rare class

F1 vs Other Vol. 19 Metrics

Use F1 when…	Use something else when…
You need one thresholded number for CI	You have not chosen a threshold yet → ROC-AUC / PR-AUC
Classes imbalanced, TN not the story	Classes balanced, symmetric cost → accuracy OK as extra
Both FP and FN matter similarly	One error dominates → P, R, or Fβ
Classification / span tagging	Free-form generation → BLEU / ROUGE / human

ROUGE reports precision, recall, and F-measure per n-gram type. Treat summary “ROUGE-L F1” the way you treat classifier F1: never without looking at the two halves. BLEU is not F1; it is geometric mean of n-gram precisions times brevity penalty—still a one-number blend of “clean” vs “too short.”

Engineering Habit — F1 Gate with Support

CI should fail if macro-F1 drops more than X or if any class with support ≥ N falls below a floor. A beautiful overall F1 with support=2 on “refund” intent is a lottery ticket, not a regression test.

Related Lectures

Lecture	Role
Accuracy	Includes TN; F1 does not
Precision / Recall	The two inputs to F1
ROC Curve	Threshold-free view before you freeze F1
BLEU / ROUGE	One-number blends for generated text
Vol. 05 Model Evaluation	First F1 in this curriculum

Common Misconception

“Highest F1 is the best model.” Only at a stated threshold, averaging, and label definition—and only if P and R costs are similar. Second: average="binary" on multiclass raises; students paste binary snippets. Third: F1 on the training set is not a ship metric (Vol. 05 test-set rule still holds after Kubernetes). Fourth: “F1 0.91 on summarization” without saying ROUGE-1 vs ROUGE-L vs classifier F1 is undefined.

Knowledge Check

Short Answer: Write F1 in terms of P and R. Answer: 2PR / (P + R) (harmonic mean).
True/False: True negatives appear in the F1 formula. Answer: False.
Multiple Choice: P=1, R=0: F1 is: (a) 0.5, (b) 1, (c) 0. Answer: (c).
Short Answer: What does β=2 in Fβ emphasize? Answer: Recall (more than precision).
True/False: Macro-F1 weights rare classes equally. Answer: True.
Multiple Choice: Single-label multiclass micro-F1 equals: (a) accuracy, (b) AUC, (c) BLEU. Answer: (a).
Short Answer: Why use harmonic mean instead of arithmetic? Answer: So a near-zero P or R cannot be hidden by the other being high.
True/False: You should tune F1 on the locked test set every epoch. Answer: False—tune on validation; test once.
Multiple Choice: Next lecture: (a) ROC Curve, (b) CUDA, (c) LoRA. Answer: (a).
Short Answer: Name one NLP metric that also reports an F-measure of overlap. Answer: ROUGE (ROUGE-1/2/L F).

Key Takeaways

F1 is the harmonic mean of precision and recall; TNs do not count.
Use Fβ when one error type should weigh more; use macro-F1 for rare classes.
Always publish P, R, support, threshold, and averaging with F1.
F1 is a thresholded CI gate; ROC comes next for ranking quality.
Next: ROC Curve.

Trainer’s Guide

Lab: Sweep thresholds, plot P, R, and F1. Students pick the F1-max threshold on validation and a separate F2-max threshold; compare test confusion matrices.

Discussion: Intent router with 12 classes. Is micro-F1 an acceptable SLO? Force the answer to include a per-class floor.

Recap: F1 compresses precision and recall into one thresholded score. See all thresholds next: ROC Curve.