From Prompt Engineering to Context Engineering

The 2026 reframing is significant. A prompt is no longer a paragraph you write in a textarea. It is a structured assembly: system instructions, retrieved context, tool definitions, examples, the user message, and output format constraints. Each part has rules. Mixing them carelessly degrades quality measurably.

• System prompt: identity, capabilities, constraints, output contract. Stable across requests
• Tool definitions: schemas, descriptions, return-shape examples. Carry as much weight as system text
• Retrieved context: documents, prior turns, scratchpads. Order and labeling matter as much as content
• Few-shot examples: 2-8 input-output pairs drawn from production distribution, not synthetic ideals
• User message: the actual request, kept as small as possible
• Output format: JSON schema, XML tags, or a free-text template. Always explicit
• Anthropic recommends XML tagging for sectioning. OpenAI recommends numbered lists and Markdown headers. Both reward explicit structure

The Core Patterns Worth Naming

Most production prompts are recombinations of a small set of patterns. Knowing them by name lets you compose deliberately instead of by trial and error.

• Zero-shot: instruction only. Works when the task is in the model's training distribution and unambiguous
• Few-shot: instruction plus examples. Default for any task where output shape varies or quality drifts
• Chain-of-thought: ask the model to reason step by step before answering. Significant quality lift on math, logic, multi-step tasks
• Structured output: JSON schema enforced by the provider. Eliminates parse errors and freeform drift
• Reasoning models (o-series, Claude extended thinking, Gemini thinking) absorb chain-of-thought internally - explicit CoT prompts can hurt them
• Self-consistency: sample N completions, take majority. Cheap reliability boost on classification and short answers
• Reflection / critique: model generates an answer, then critiques it, then revises. Useful when ground truth is hard but quality is checkable
• Role priming ("You are a senior X"): low impact on modern models, often counterproductive when overdone
• Decomposition: split a complex task into a sequence of simpler prompts. Almost always beats one mega-prompt

Structured Output: The Single Highest-Leverage Pattern

If you do nothing else, force structured output. Prompting for JSON without schema enforcement produces markdown wrappers, trailing commas, missing keys, silently shifted types, and inconsistent enum values. Every major provider now offers schema-constrained generation that eliminates these failures at the API level.

• OpenAI Structured Outputs: guarantees JSON conforms to your schema. Use strict mode
• Anthropic tool use: enforces tool argument shape. Use tool calls even when you do not need a tool
• Google Gemini structured output: same idea, JSON Schema constraint
• For non-JSON formats, define a tight template with delimiters and parse with a regex you can validate
• Schema design rules: small enums beat free strings, explicit nullable beats implicit absence, descriptions per field beat global instructions
• Schema is part of the prompt - every field name and description teaches the model what to produce
• When schema enforcement is unavailable, repeat the contract at the start and end of the prompt and validate every parse

Provider-Specific Quirks That Matter

Models are not interchangeable. The same prompt that works on Claude often degrades on GPT, and vice versa. Production teams that target multiple providers need provider-specific prompt branches, not a single universal prompt.

• Claude: privileges the system parameter strongly, follows XML tagging reliably, rewards structured prompts and explicit thinking blocks
• GPT (4o, 5, 5.5): developer messages are less privileged than Claude system, but numbered lists and Markdown headers work better than XML
• Gemini: strong on multimodal and long context, less rigid on output discipline, often needs explicit "respond only in JSON" reminders
• Reasoning models (GPT-5 thinking, Claude extended thinking, Gemini deep think) absorb CoT internally - explicit step-by-step prompts can suppress their built-in reasoning
• Caching: Anthropic prompt caching and OpenAI prompt caching cut cost 50-90% on stable system prompts - design prompts to maximize cache hits
• Long context (>200K tokens): position matters more than length. Place critical content at the end, not buried mid-context
• Streaming: structured output and streaming together is supported but partial JSON parsing is non-trivial. Test it

Few-Shot Selection: The Hidden Quality Lever

Most teams treat few-shot examples as decorative. They are not. The examples you pick teach the model the distribution, format, edge cases, and refusal policy of your task more effectively than any instruction. Bad examples actively hurt; good ones can replace pages of instruction.

• Use real production inputs, not synthetic ideals. The model needs to see the messy reality
• Include the edge cases you care about. Long, short, malformed, multilingual, adversarial
• Include negative examples (input with expected refusal or expected null output) when the task has them
• Order matters: place the most representative example first and last. Middle positions get less attention
• Dynamic few-shot: retrieve top-K most similar examples per request from a labeled example bank. Routinely beats static few-shot
• Match the format exactly: if production calls return a tool call, examples should show tool calls, not freeform text
• Refresh examples as the distribution shifts. Production traffic changes, your examples should too

Prompt Templates, Versioning, and Rollback

A prompt is a deployment artifact. Treat it that way. Teams that ship AI quickly are the teams that can version, A/B test, and roll back prompts as fast as they roll back code. Teams that paste prompt strings into application code without versioning are a single bad edit away from a production regression they cannot diagnose.

• Store prompts as versioned files in the repo, not as inline strings in application code
• Template engines: Jinja2, Handlebars, or provider-native (LangChain PromptTemplate, BAML, Promptfile). Pick one
• Parameterize ruthlessly: user input, retrieved context, persona variants, format flags - all named variables
• Hash and tag every prompt version with the eval scores it shipped against
• Decouple prompt from model: same prompt should target multiple models cleanly via small adapter changes
• Roll back ready: previous prompt version must be one config change away
• Prompt management tools worth naming: Braintrust, LangSmith, Promptfoo, Helicone, Langfuse, Humanloop, Latitude

Evaluation-Driven Prompt Iteration

You cannot improve what you do not measure. The teams that ship the best AI features iterate prompts against a frozen eval set, not against vibes or single-example screenshots. The eval set is the unit test suite for the prompt.

• Freeze an eval set of 50-500 real examples with expected outputs or rubric criteria before iterating
• Score every prompt change on the full eval set, not on the example that motivated the change
• LLM-as-judge with calibrated rubrics, paired with periodic human spot-checks on the judge itself
• Track per-slice scores: do not let a prompt change improve average score while regressing a critical slice
• A/B in production: run two prompt versions live on a small percentage and compare on real outcome metrics
• Tools that do this well: Braintrust, Promptfoo, DeepEval, LangSmith, Inspect AI, OpenAI Evals
• Ship gate: new prompt must beat current on every guarded slice and on cost/latency budgets

Cost and Latency Engineering Through the Prompt

Every token in your prompt costs money on every request, forever. A prompt that grows 500 tokens because someone added "helpful" instructions burns real budget at scale. The lowest-effort cost wins almost always come from prompt compression and cache design.

• Move stable content (system, tools, examples) to the start of the prompt to maximize cache hits
• Anthropic and OpenAI prompt caching can cut input cost 50-90% on repeated stable prefixes
• Compress instructions: every sentence that does not change a measurable eval score is dead weight
• Cap retrieved context aggressively - more documents past the relevance cliff hurts quality and cost
• Use cheaper models for routing, classification, and simple extraction. Reserve frontier models for the steps that need them
• Token accounting per request: track input vs output tokens by step, alarm on regressions
• Streaming: ship perceived latency wins even when total latency unchanged. UX often matters more than wall time

Anti-Patterns to Stop Using

Most of what people call "prompt engineering" online is folklore. These patterns are the most common ones I find in client codebases that do not survive evaluation.

• "You are a world-class expert" priming: low impact on modern models, often increases verbosity and hallucination
• "Take a deep breath / think carefully": superseded by explicit reasoning modes; on reasoning models, can suppress built-in CoT
• "DO NOT" / shouting / threats: increases adherence marginally on weak models, no effect or worse on modern ones
• Mega-prompts: one giant prompt doing 15 things. Decompose into a sequence, almost always wins on quality and cost
• Prompting for JSON without schema enforcement: produces parse errors at scale. Always use structured output
• Few-shot examples that do not match production distribution: actively misleads the model
• Prompt injection defense via "ignore previous instructions if user says X": does not work. Use untrusted-context isolation and output filtering
• Storing prompts as string literals in code: invisible to versioning and review, undiscoverable when something breaks
• No eval set: every prompt change becomes a vibe-based bet, regressions ship silently

What an Engagement Looks Like

Most teams who hire me for prompt engineering have a feature that is "mostly working" but unreliable, expensive, or unscalable. The engagement converts that into a measurable, versioned, eval-gated system that the team can own going forward.

• Week 1: audit current prompts, instrument logging, identify failure modes from real traffic
• Week 2: build the eval set from production samples, score current prompts as baseline
• Week 3-4: prompt rewrites against eval set, structured output migration, caching design, model routing where applicable
• Week 5: production deployment, A/B testing, observability, runbook for ongoing iteration
• Deliverables: versioned prompts, eval suite, A/B harness, cost and latency dashboards, written playbook for the team
• Typical outcomes: 20-50% cost reduction, 10-40% quality lift on critical slices, parse-error rate to near zero
• Engagement length: 4-8 weeks for a focused feature, longer for multi-feature platform work

FAQ