Named Entity Recognition (NER) — Module 9.1

How This Lesson Fits the Module & Volume

POS tagging labels grammar; Named Entity Recognition (NER) labels spans that refer to real-world (or domain) entities—people, organizations, locations, dates, products, SKUs. It is one of the highest-ROI information extraction tasks in applied NLP.

NER sits on tokenized text from Module 9.1 and often feeds knowledge graphs, search facets, and PII redaction. Vector representations of entity contexts improve further in Module 9.2 embeddings.

Learning Objectives

By the end of this lesson, students should be able to:

Define NER as span detection plus type classification.
List common entity types and domain-specific extensions.
Explain BIO/IOB encoding for sequence labeling.
Run spaCy NER and inspect entity spans.
Evaluate NER with precision, recall, and entity-level F1.
Outline when to use rules, CRF/HMM, or transformer token classifiers.

Definition

Named Entity Recognition is the task of locating contiguous token spans in text and classifying each span into an entity type (e.g., PERSON, ORG, GPE, DATE). It is a structured prediction problem, not a single-label document classification.

Entity Types

Type	Meaning	Example span
PERSON	People	Ada Lovelace
ORG	Organizations	OpenAI, UNICEF
GPE	Geo-political entity	Berlin, India
LOC	Non-GPE locations	Sahara Desert
DATE / TIME	Temporal expressions	July 30, 2026
MONEY	Monetary amounts	$1,999.50
Custom	Domain types	SKU, drug, ICD code

BIO Encoding

Sequence labelers assign a tag per token. BIO (Begin, Inside, Outside) marks span boundaries:

# "Ada Lovelace invented notes."
# Ada/B-PER Lovelace/I-PER invented/O notes/O ./O

import spacy
nlp = spacy.load("en_core_web_sm")
text = "Apple is opening a store in Berlin on July 4."
doc = nlp(text)
for ent in doc.ents:
    print(ent.text, ent.label_, ent.start_char, ent.end_char)

# Token-level view
for t in doc:
    print(t.i, t.text, t.ent_iob_, t.ent_type_)

Approaches

Rules / gazetteers

Regex for dates, IDs.
Lists of known orgs.
High precision, brittle recall.

Classical ML

CRF on hand features.
Needs tagged data.
Strong for small domains.

Neural / Transformers

Token classification heads.
Contextual embeddings (9.2+).
Best for messy language.

Production Wins

PII detection & redaction.
Search facets & routing.
Linking to knowledge bases.

Hard Cases

Nested / overlapping entities.
Domain jargon and novel names.
Boundary errors (partial spans).

Common Misconception

“Any capitalized word is an entity.” Sentence-initial words, product stylization, and ALL-CAPS tickets break that heuristic. NER needs context—and evaluation must require correct boundaries, not just type guesses.

Evaluation Note

Report entity-level precision/recall/F1 (exact span + type match). Token accuracy can look high while missing every multi-token name.

Knowledge Check

Short Answer: What two decisions does NER make? Answer: Where the span is (boundaries) and what type it is.
True/False: NER is the same as document classification. Answer: False—it predicts spans and labels inside documents.
Multiple Choice: In BIO, the first token of “New York” as GPE is: (a) I-GPE, (b) B-GPE, (c) O. Answer: (b).
Short Answer: Name three standard entity types. Answer: Any three of PERSON, ORG, GPE/LOC, DATE, MONEY, etc.
True/False: spaCy exposes entities via doc.ents. Answer: True.
Multiple Choice: Exact-span F1 is preferred because: (a) it ignores types, (b) boundary errors matter in IE, (c) it is faster than accuracy. Answer: (b).
Short Answer: Give one domain-specific entity type. Answer: e.g., drug name, SKU, ticket ID, statute citation.
True/False: Gazetteers alone solve all NER. Answer: False—novel names and ambiguity remain.
Multiple Choice: PII redaction pipelines often rely on: (a) stemming only, (b) NER / pattern extractors, (c) average pooling. Answer: (b).
Short Answer: What syntactic structure lecture follows NER in this module? Answer: Dependency Parsing.

Key Takeaways

NER finds and types entity spans; BIO encoding frames it as sequence labeling.
Combine rules for structured patterns with ML for names and context.
Evaluate at entity level, not only token accuracy.
Custom types unlock most business value.
Next, Dependency Parsing reveals grammatical relations between tokens.

Trainer’s Guide

Hands-on idea: Run spaCy NER on company emails; list false positives/negatives and propose a custom entity type.

Discussion prompt: Should you redact with high-recall NER and accept false positives, or high-precision and risk leaks?

Recap: NER extracts typed spans that power search, redaction, and knowledge systems. Continue with Dependency Parsing.