Data Annotation — Module 4.1

How This Lesson Fits the Module

Data Labeling assigns whole-example targets (class, score, ranking). Data annotation marks structure inside the example: bounding boxes on images, named entities in text, polygon segments in vision, token spans for instruction tuning.

Computer vision, NLP, and multimodal models consume annotation formats (COCO, CoNLL, JSONL with spans). ML engineers must serialize, validate, and version these artifacts—not only click tools in a UI.

Learning Objectives

By the end of this lesson, students should be able to:

Contrast classification labels with spatial and token-level annotations.
Represent bounding boxes in pixel and normalized coordinates.
Export NER annotations as BIO/IOB2 token tags or span JSON.
Compare instance segmentation, semantic segmentation, and detection tasks.
Validate annotation files against schema before training.
Choose tooling (Label Studio, CVAT, custom) for team workflow.

Introduction: Labels vs Annotations

Term	Granularity	Example	Typical Model
Label	Whole example	Image → “cat”	Image classifier
Detection annotation	Regions + classes	Box around each pedestrian	Object detector (YOLO, Faster R-CNN)
NER annotation	Token spans	`ACME` → ORG	Token classifier / span model
Segmentation mask	Per-pixel class	Road vs sidewalk	U-Net, Mask R-CNN
Relation / event	Graph on entities	Person works_at Org	Joint extraction models

Definition — Bounding Box

A bounding box is an axis-aligned rectangle (x_min, y_min, x_max, y_max) or (x, y, width, height) enclosing an object. Production pipelines often store normalized coordinates (0–1 relative to image width/height) so annotations survive resizing.

Bounding Boxes for Object Detection

# COCO-style record (simplified)
annotation = {
    "image_id": "img_0042",
    "category": "defect",
    "bbox_xywh": [120, 80, 64, 48],  # x, y, width, height
    "area": 64 * 48,
}

def normalize_bbox(bbox, img_w, img_h):
    x, y, w, h = bbox
    return [x / img_w, y / img_h, w / img_w, h / img_h]

Detection

Boxes + class per object
Multiple objects per image
Metrics: mAP, IoU

Segmentation

Pixel masks or polygons
Instance vs semantic tasks
Heavier storage and labeling cost

Named Entity Recognition (NER)

NER assigns types to spans in text: persons, organizations, locations, product SKUs. Annotations can be span JSON or per-token tags.

Definition — BIO / IOB2 Tagging

BIO tagging labels each token: B-ORG (begin entity), I-ORG (inside), O (outside). IOB2 requires that multi-token entities start with B-. This format trains classic CRF and transformer token classifiers.

# Span JSON — common in modern LLM fine-tuning pipelines
record = {
    "text": "Contact ACME Corp in Berlin.",
    "entities": [
        {"start": 8, "end": 17, "label": "ORG"},
        {"start": 21, "end": 27, "label": "LOC"},
    ],
}

import pandas as pd
df = pd.DataFrame([
    {"doc_id": 1, "start": 8, "end": 17, "label": "ORG"},
    {"doc_id": 1, "start": 21, "end": 27, "label": "LOC"},
])
# Validate: 0 <= start < end <= len(text)

Volume 03 Bridge Use regex for weak labels (e.g. email-shaped tokens as B-CONTACT) but validate with human annotation before trusting metrics. Regex bootstraps; annotations ground truth.

Segmentation vs Labeling

Semantic segmentation assigns a class per pixel without distinguishing object instances (all road pixels = class 1). Instance segmentation separates individual objects (car #1 vs car #2). Labeling alone (“scene = street”) cannot train mask heads—you need polygon or raster annotations.

Task	Annotation Cost	Output Artifact
Image classification	Low	CSV label column
Object detection	Medium	COCO JSON, YOLO txt
Instance segmentation	High	Polygon lists or RLE masks
Semantic segmentation	High	PNG mask or color map
NER	Medium	CoNLL, spaCy DocBin, JSONL spans

Validation Before Training

def validate_spans(text: str, entities: list) -> None:
    n = len(text)
    for e in entities:
        s, end = e["start"], e["end"]
        assert 0 <= s < end <= n, f"bad span {e}"
        assert e["label"], "empty label"

def validate_bbox(bbox, w, h):
    x, y, bw, bh = bbox
    assert 0 <= x < w and 0 <= y < h
    assert bw > 0 and bh > 0
    assert x + bw <= w and y + bh <= h

Common Misconception: “Annotation tools export production-ready datasets automatically.”

Reality: Exports need schema validation, class balance checks, train/val leakage scans (near-duplicate images), and version pinning.

Common Misconception: “NER and object detection use the same metrics as classification accuracy.”

Reality: Span F1 and mAP@IoU account for localization—a wrong boundary is a failure even if the class name is right.

Tooling Landscape

Label Studio, CVAT, and Prodigy support multiple modalities. For large vision projects, teams often combine model-assisted pre-annotation with human correction. Store raw tool exports in the raw layer; convert to trainer-specific formats in ETL.

Knowledge Check

Short Answer: Label vs annotation? Answer: Label is whole-example target; annotation marks internal structure.
True/False: Normalized bounding boxes help when images are resized. Answer: True.
Multiple Choice: BIO tag for first token of an entity: (a) O, (b) I-ORG, (c) B-ORG, (d) END. Answer: (c).
Short Answer: What does IoU measure in detection? Answer: Overlap between predicted and ground-truth boxes.
True/False: Semantic segmentation distinguishes car instances. Answer: False (instance segmentation does).
Multiple Choice: Span JSON uses: (a) character offsets, (b) only bag-of-words, (c) RGB pixels, (d) SQL joins. Answer: (a).
Short Answer: Why validate start < end on NER spans? Answer: Prevents empty or out-of-bounds annotations breaking trainers.
True/False: COCO JSON is common for detection datasets. Answer: True.
Multiple Choice: Convert tool exports to training format in: (a) labeling UI only, (b) ETL, (c) loss function, (d) GPU driver. Answer: (b).
Short Answer: One advantage of model-assisted pre-annotation? Answer: Faster human correction vs drawing every box from scratch.

Key Takeaways

Annotations are structured targets inside examples—boxes, spans, masks.
Use normalized coordinates and validated span offsets in stored artifacts.
Segmentation tasks need pixel/polygon labels, not whole-image classes alone.
Validate exports before training; tool output is not automatically clean.
Next: ETL to orchestrate extract-transform-load at scale.

Trainer’s Guide

Dual-modality lab: Annotate 10 images with boxes and 10 sentences with NER spans. Students write validators and export one unified JSONL manifest linking file paths to annotations.

Metric moment: Show how a box shifted 5 pixels drops IoU—motivates tight guidelines from the labeling lecture.

What’s Next Orchestrate collection, cleaning, labeling, and annotation into pipelines in ETL (Extract, Transform, Load).