← Master Index
Vol. 12 Module 12.4 Lecture

Prompt Tuning

Fine-Tuning Deep Dive

How This Lesson Fits the Module & Volume

Prefix tuning injects soft tokens deep in every layer. Prompt tuning is the lighter cousin: only a few trainable embeddings prepended at the input embedding layer. It is among the cheapest PEFT methods and a bridge to classic prompt design before the module capstone on adapters.

Learning Objectives

By the end of this lesson, students should be able to:

  • Define prompt tuning (soft prompts at the input).
  • Contrast soft prompts with discrete hand-written prompts.
  • Compare prompt vs prefix tuning on depth and parameter count.
  • Configure PromptTuningConfig in PEFT.
  • Discuss init strategies (random vs vocab-based).
  • Know when soft prompts underfit relative to LoRA.
Definition

Prompt tuning freezes the entire pretrained model and learns a small matrix of continuous prompt embeddings P ∈ &mathbb;Rn×d that are prepended to the embedded input tokens. Only P is trained (n soft tokens). The model reads them like ordinary embeddings but they need not correspond to real vocabulary strings.

Soft vs Discrete vs Prefix

ApproachTrainable?WhereParams
Discrete promptingNo (human text)Input tokens0
Prompt tuningYes (embeddings)Input onlyn × d
Prefix tuningYes (prefixes)Every layer KVLarger
LoRAYes (A, B)Selected weightsDepends on r
Embed input

Token → vectors

Prepend P

n soft embeddings

Frozen LM

Standard forward

Update P

Backprop only prompts

PEFT Prompt Tuning

from transformers import AutoModelForCausalLM from peft import PromptTuningConfig, PromptTuningInit, get_peft_model, TaskType model = AutoModelForCausalLM.from_pretrained("gpt2") cfg = PromptTuningConfig( task_type=TaskType.CAUSAL_LM, num_virtual_tokens=8, prompt_tuning_init=PromptTuningInit.TEXT, prompt_tuning_init_text="Classify the sentiment of the following text:", tokenizer_name_or_path="gpt2", ) model = get_peft_model(model, cfg) model.print_trainable_parameters() # After training, soft prompts are task-specific; base GPT-2 unchanged on disk

When Prompt Tuning Shines

Good fit

  • Huge frozen models.
  • Many tasks, tiny storage.
  • Classification-style heads.

Weak fit

  • Heavy style/domain shift.
  • Needs weight-space capacity.
  • Very long soft n (context tax).

Init tip

  • TEXT init often helps.
  • Random can work with care.
  • Match tokenizer carefully.

Strengths and Tradeoffs

Strengths

  • Extremely small trainable sets.
  • Simple serving: cache base, swap P.
  • Elegant link to prompting intuition.

Tradeoffs

  • Less expressive than LoRA on hard SFT.
  • Consumes context window.
  • Soft tokens are not interpretable text.
Common Misconception

“Prompt tuning means hiring better prompt engineers.” Engineering discrete prompts is complementary. Prompt tuning is gradient-based learning of soft embeddings—an optimization method, not a writing workshop.

Knowledge Check

  1. Short Answer: What is trained in prompt tuning? Answer: A small set of continuous input embeddings (soft prompts).
  2. True/False: The backbone weights update freely in prompt tuning. Answer: False—they stay frozen.
  3. Multiple Choice: Soft prompts live primarily: (a) at the input embedding layer, (b) in the optimizer only, (c) in CSS. Answer: (a).
  4. Short Answer: How does prompt tuning differ from prefix tuning? Answer: Input-only vs per-layer prefixes (shallower vs deeper).
  5. True/False: Soft prompts always decode to readable English words. Answer: False.
  6. Multiple Choice: TEXT initialization: (a) seeds prompts from a string, (b) deletes the tokenizer, (c) trains CNNs. Answer: (a).
  7. Short Answer: Name one downside versus LoRA for hard domain SFT. Answer: Less capacity / may underfit.
  8. True/False: Virtual tokens reduce remaining context for the user text. Answer: True.
  9. Multiple Choice: PEFT class: (a) PromptTuningConfig, (b) AvgPool1d, (c) DynamicBatchCSS. Answer: (a).
  10. Short Answer: What Vol. 12 capstone lecture is next? Answer: Adapters.

Key Takeaways

  • Prompt tuning learns soft input embeddings only.
  • Cheapest PEFT tier; great multi-task storage story.
  • Shallower than prefix tuning; weaker than LoRA on hard shifts.
  • Distinct from discrete prompt engineering.
  • Next (capstone): Adapters.
Trainer’s Guide

Hands-on idea: Compare accuracy of discrete few-shot prompts vs 8-token prompt tuning on a classification LM.

Discussion prompt: Should product teams expose soft prompts to end users, or keep them internal artifacts?

Recap: Prompt tuning adapts frozen models with a handful of learned input embeddings. Finish the module with Adapters.