← Master Index
Vol. 13 Module 13.1 Lecture

System Prompt

Prompting Techniques

How This Lesson Fits the Module & Volume

Chat-style models separate durable policy from the live request. The system prompt (system message) holds identity, safety, format contracts, and tool rules that should not change every turn. The next lecture covers the complementary user prompt.

System prompts often encode earlier techniques—roles, JSON schemas, and Markdown skeletons—so products stay consistent across sessions.

Learning Objectives

By the end of this lesson, students should be able to:

  • Define the system prompt and distinguish it from user/assistant turns.
  • Place policy, persona, and output contracts in the system message.
  • Keep system prompts stable, versioned, and testable.
  • Avoid stuffing volatile task data into the system role.
  • Coordinate system rules with guardrails outside the model.
  • Debug conflicts between system and user instructions.
Definition

A system prompt is the high-priority instruction message (role system in chat APIs) that sets the assistant’s standing behavior—persona, policies, tools, and output format—across a conversation, independent of any single user query.

What Belongs in System vs. User

LayerPut hereAvoid here
SystemPersona, safety, format, tool policyHuge one-off documents
UserQuestion, task inputs, per-request contextContradicting core safety
AssistantPrior model replies (history)Secret credentials
Identity

Who the assistant is.

Rules

Do / don’t policies.

Format

JSON / MD / tools.

Escalate

When to refuse / hand off.

Good system content

  • Brand voice
  • Refusal categories
  • Schema reminder

Move to user

  • Today’s ticket text
  • Retrieved chunks
  • One-shot examples per task

Outside the model

  • Auth & quotas
  • PII filters
  • Hard blocklists

Practical API Example

SYSTEM_PROMPT = """ You are Nova Support, a concise B2B product assistant. Rules: - Answer only from provided when present; otherwise say you lack data. - Never invent order IDs, prices, or legal advice. - If asked for disallowed content, refuse briefly and offer a safe alternative. Output: - Default to short Markdown with ## Answer and ## Sources. - When the user asks for machine output, return ONLY JSON matching the requested schema. """ messages = [ {"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": f"\n{rag_chunks}\n\n\nQuestion: {question}"}, ] # Version the system string in git (e.g. prompts/system_v3.txt) and log prompt_version per request.

Strengths

  • Consistent behavior across turns
  • Central place for policy & format
  • Easier product-wide updates

Tradeoffs

  • Overlong systems waste tokens
  • Conflicts with clever user overrides
  • Not a substitute for external guardrails
Common Misconception

“The system prompt is an unbreakable security boundary.” Models can still be steered by adversarial user content. Treat system text as soft policy; enforce hard safety with guardrails, filters, and privilege separation.

Knowledge Check

  1. Short Answer: What is a system prompt? Answer: The standing chat message that sets durable assistant behavior/policy.
  2. True/False: One-off ticket text usually belongs in the system role. Answer: False—put it in the user message.
  3. Multiple Choice: System prompts commonly hold: (a) persona and format rules, (b) GPU firmware, (c) raw weight tensors. Answer: (a).
  4. Short Answer: Why version system prompts? Answer: To reproduce behavior and attribute eval/regression changes.
  5. True/False: System prompts alone guarantee safety against injection. Answer: False.
  6. Multiple Choice: Auth and quotas should live: (a) outside the model, (b) only in witty system jokes, (c) in temperature. Answer: (a).
  7. Short Answer: Name a conflict to watch for. Answer: User instructions that contradict system safety/format rules.
  8. Short Answer: What should you log with each call? Answer: prompt_version (and related config IDs).
  9. Multiple Choice: Overlong system prompts mainly risk: (a) token cost and dilution, (b) faster GPUs, (c) free context. Answer: (a).
  10. True/False: The next lecture covers the per-request user prompt. Answer: True.

Key Takeaways

  • System prompts hold durable identity, policy, and format.
  • Keep volatile task data in the user turn.
  • Version and evaluate; back with external guardrails.
  • Next: User Prompt.
Trainer’s Guide

Hands-on: Split a messy all-in-one prompt into system vs. user; compare consistency over 10 queries.

Discussion: How short can a production system prompt be while still encoding brand + safety?

Recap: The system prompt is your product’s standing contract with the model. Continue with User Prompt.