Amazon Transcribe — Module 16.2

How This Lesson Fits the Module & Volume

Google and Azure cover two hyperscalers. Amazon Transcribe is AWS-native ASR: S3 in, JSON out, IAM everywhere, plus first-party Call Analytics and Transcribe Medical. If recordings already land in S3 and the rest of the app is Lambda/Step Functions, Transcribe is the default—even when Deepgram might win a pure latency bake-off.

This closes the hyperscaler STT trio before the module shifts to generative voice (ElevenLabs, PlayHT) and then cross-cutting diarization and real-time design. Capability background: 16.1 STT.

Learning Objectives

By the end of this lesson, students should be able to:

Describe Amazon Transcribe batch (S3 jobs) vs streaming (WebSocket / HTTP2).
Start a transcription job with speaker labels and PII redaction via boto3.
Place Call Analytics and Transcribe Medical as specialized SKUs, not free flags.
Use custom vocabularies / vocabulary filters as bias and suppression tools.
Compare AWS vs Azure vs Google vs Deepgram/Whisper on streaming, languages, pricing, on-prem.
Design a pipeline: S3 → Transcribe → redacted JSON → downstream LLM / warehouse.

Definition

Amazon Transcribe is AWS’s managed automatic speech recognition service. Batch jobs read audio from S3 and write transcript JSON (optionally with speaker labels, alternative hypotheses, and PII redaction). Streaming APIs return partial and final transcripts for live audio. Adjacent products—Transcribe Medical and Transcribe Call Analytics—add domain models and conversation intelligence on the same audio plane.

AWS Data Plane First

The “why Transcribe” argument is rarely a single WER chart. It is KMS-encrypted buckets, VPC endpoints, CloudTrail, and one IAM role from the telephony recorder to the transcript. Sending the same bytes to AssemblyAI may be simpler for summaries, but it is a second processor under privacy review. Decide processor count before you decide model brand.

Catalog Snapshot (Qualitative)

Dimension	Amazon Transcribe	Azure Speech	Google STT	Deepgram / Whisper
Accuracy posture	Solid general ASR; Medical and Call Analytics are separate eval targets	General + Custom Speech	Chirp-class + domain models	Whisper multilingual; Deepgram live conversational
Streaming	Yes (streaming Transcribe)	Speech SDK	gRPC streaming	Deepgram WS-first; Whisper no
Languages	Broad; confirm LID, medical, and analytics per locale	Very broad	Very broad	Whisper very broad
Pricing posture	Typically per-second metering + extra for PII, analytics, medical—read the SKU sheet	Hours / characters	Per-15s tiers	Per-minute or GPU
On-prem vs cloud	Cloud; PrivateLink / VPC endpoints (not a Whisper-style air-gap binary)	Containers available	On-prem GKE option	Whisper = on-prem weights
Diarization	`ShowSpeakerLabels` + max speakers	Conversation transcription	Diarization config	Flags / pyannote
PII / intel	Content redaction; Call Analytics (categories, sentiment, issues)	Masking + Cognitive extras	DLP on text	AssemblyAI intel specialist

Batch vs Streaming vs Call Analytics

Batch job

S3 URI in
Poll job status
JSON transcript URI out
Archives, podcasts, QA

Streaming

Live PCM/audio frames
Partial + final events
Agent assist, captions
See lecture 10 for design

Call Analytics / Medical

Paid specialized APIs
Categories, sentiment, PII
Clinical vocabularies
Separate accuracy + BAA review

Python: Start a Batch Job with Speakers and PII Redaction

The job is asynchronous. Do not busy-wait in a web request—use EventBridge / Step Functions when a job completes. Vocabulary filters suppress profanity or unwanted terms; custom vocabularies boost jargon. Both can hurt WER if misused—eval.

import time
import boto3

client = boto3.client("transcribe", region_name="us-east-1")
job_name = "call-2026-08-13-001"

client.start_transcription_job(
    TranscriptionJobName=job_name,
    Media={"MediaFileUri": "s3://your-calls/inbound/001.wav"},
    MediaFormat="wav",
    LanguageCode="en-US",  # or IdentifyLanguage=True where supported
    OutputBucketName="your-transcripts",
    OutputEncryptionKMSKeyId="arn:aws:kms:us-east-1:123:key/your-key",
    Settings={
        "ShowSpeakerLabels": True,
        "MaxSpeakerLabels": 4,
        "ShowAlternatives": True,
        "MaxAlternatives": 3,
        # "VocabularyName": "product-sku-vocab",
        # "VocabularyFilterName": "profanity-filter",
        # "VocabularyFilterMethod": "mask",
    },
    ContentRedaction={
        "RedactionType": "PII",
        "RedactionOutput": "redacted",  # or redacted_and_unredacted if policy allows
        "PiiEntityTypes": ["SSN", "CREDIT_DEBIT_NUMBER", "NAME", "PHONE"],
    },
)

while True:
    desc = client.get_transcription_job(TranscriptionJobName=job_name)
    status = desc["TranscriptionJob"]["TranscriptionJobStatus"]
    if status in ("COMPLETED", "FAILED"):
        break
    time.sleep(8)

if status != "COMPLETED":
    raise RuntimeError(desc["TranscriptionJob"].get("FailureReason"))
print(desc["TranscriptionJob"]["Transcript"]["TranscriptFileUri"])

Streaming Sketch

AWS streaming uses a signing-heavy WebSocket or HTTP/2 event stream. Prefer the official streaming SDK / Transcribe Streaming parser rather than hand-rolling SigV4. Partial results behave like Deepgram/Azure: UI vs system of record. Details in real-time transcription.

# Teaching pattern only — use amazon-transcribe streaming SDK in production
# 1) Open signed streaming session (language, media encoding, sample rate)
# 2) Send audio chunks (e.g. 16 kHz PCM)
# 3) On TranscriptEvent: if result.IsPartial: update UI else commit text
#
# from amazon_transcribe.client import TranscribeStreamingClient
# client = TranscribeStreamingClient(region="us-east-1")
# stream = await client.start_stream_transcription(
#     language_code="en-US", media_sample_rate_hz=16000, media_encoding="pcm",
# )

Pick Amazon Transcribe when

Audio already lives in S3 / Kinesis Voice.
You need IAM, KMS, CloudTrail, PrivateLink.
Call Analytics or Transcribe Medical is the product.
Batch jobs fit Step Functions better than a third-party webhook.

Pick something else when

Air-gap / open weights → Whisper.
Microsoft estate → Azure Speech.
Lowest-effort live WS outside AWS → Deepgram.
Rich LLM meeting notes without AWS analytics → AssemblyAI.

Common Misconception

“Transcribe Medical is just LanguageCode=en-US with a stethoscope emoji.” Medical is a distinct model/SKU with its own vocabulary, pricing, and compliance scope. Enabling speaker labels or PII redaction on the general API does not give you clinical-grade term accuracy. Use the medical path only after a clinical eval and legal review—not as a checkbox.

Knowledge Check

Short Answer: What is the standard batch I/O pattern for Transcribe? Answer: Audio in S3, async job, transcript JSON written to S3 (or a transcript URI).
True/False: ShowSpeakerLabels identifies speakers by legal name automatically. Answer: False—it diarizes anonymous speaker labels.
Multiple Choice: PII content redaction acts on: (a) CloudFront CDN cache, (b) transcript entities (redacted output), (c) EC2 instance types. Answer: (b).
Short Answer: Why might a team choose Transcribe over Deepgram despite similar live WER? Answer: Stay inside the AWS IAM/KMS/VPC processor boundary.
True/False: Call Analytics features are always included at no extra cost in every Transcribe job. Answer: False—they are specialized (usually extra) SKUs.
Multiple Choice: AWS on-prem posture vs Whisper: (a) Transcribe is cloud/VPC; Whisper weights can air-gap, (b) Transcribe ships as a laptop binary, (c) Whisper requires S3. Answer: (a).
Short Answer: Name one risk of an overly aggressive vocabulary filter. Answer: It can mask or drop legitimate domain terms and inflate effective error.
True/False: Streaming Transcribe still distinguishes partial vs final transcripts. Answer: True.
Multiple Choice: Pricing posture is closest to: (a) per-second + feature add-ons, (b) per-image diffusion, (c) free unlimited medical. Answer: (a).
Short Answer: After hyperscaler STT, which lecture starts generative voice (TTS/STT)? Answer: ElevenLabs (STT/TTS).

Key Takeaways

Amazon Transcribe is AWS-native batch (S3) and streaming ASR, plus Medical and Call Analytics SKUs.
Speaker labels, custom vocabularies, filters, and PII redaction are request settings—eval each.
Choose Transcribe when the AWS control plane is the constraint; choose specialists when latency or intel UX is the product.
Not an air-gap engine; Whisper remains the open on-prem default.
Next: ElevenLabs (STT/TTS).

Trainer’s Guide

Lab: Put a WAV on S3, start a job with speaker labels and PII redaction, download JSON, and map speaker turns to a simple timeline. Compare redacted vs unredacted only in a secure scratch space.

Architecture: Draw EventBridge → Lambda on Transcribe Job State Change → DLP → warehouse. Contrast with AssemblyAI webhooks. Include compliance processors.

Recap: Amazon Transcribe is S3-native ASR with optional analytics and medical SKUs. Continue with ElevenLabs.