Last time, Rae built a working evaluation harness — a reproducible loop that runs her support bot’s prompts against a test set, scores each response with a scorer function, and aggregates the results into a single number she can trust. She could compare two models side by side, estimate costs before a big run, and stop guessing whether a change actually helped or hurt. Now that she had hard numbers, they told her something uncomfortable: her support bot’s answers were technically present but buried in clunky, verbose, sometimes unhelpful language. The model wasn’t broken. Her prompts were.

Have you ever asked an LLM to “write a report on sales data” only to get a generic, five-paragraph essay that reads like a high school homework assignment? You know the model is capable of more, but it feels like you’re speaking two different languages.

Communication Problems, Not Model Problems

The catch is that prompt engineering isn’t about finding “magic words.” It’s about reducing ambiguity. A vague prompt forces the model to guess your intent, tone, and format. Most prompt failures aren’t the model’s fault — they’re communication problems. If you told a new intern to “do the data thing,” you wouldn’t be surprised when they come back with something useless.

Rae sees this every time she runs her harness:

• A prompt like “answer the customer’s question” returns a rambling paragraph that buries the return policy three sentences in.
• Add “be helpful” and the bot piles on so much empathy that the customer can’t find the answer.

Her harness scores confirm it: the contains-scorer passes because the right words are in there somewhere, but the relevance scores tank. She’s been tweaking prompts by gut feel, and now she has the data to prove her instincts aren’t enough. She needs patterns that move the needle — and a way to measure whether they work.

So let’s see what happens when we compare a vague prompt to a structured one.

# Scenario: Analyzing a small list of customer feedback
feedback_data = [
    "The app is slow but the support team was helpful.",
    "I hate the new UI, it's confusing.",
    "Great price, but missing the export feature."
]

# Vague Prompt
prompt_vague = f"Analyze this feedback: {feedback_data}"
# Output: "The feedback shows a mix of positive and negative sentiments regarding speed, UI, and features."

# Structured Prompt
prompt_structured = f"""
Act as a Product Operations Analyst.
Task: Categorize the following customer feedback into 'Technical Issue', 'UI/UX', or 'Feature Request'.
Format: Return a bulleted list.
Data: {feedback_data}
"""
# Output: 
# - Technical Issue: The app is slow...
# - UI/UX: I hate the new UI...
# - Feature Request: Great price, but missing the export feature...

This block contrasts a vague prompt against a structured one — the core lesson Rae needs for her support bot.

• feedback_data — a Python list of three strings, each a piece of customer feedback.
• prompt_vague = f"Analyze this feedback: {feedback_data}" — an f-string that interpolates the entire list directly into the prompt. This dumps the raw Python list representation (with brackets and quotes) into the text, which is messy but functional. The comment on the next line shows the kind of generic, unactionable output this produces.
• prompt_structured = f"""...""" — a triple-quoted f-string for a multi-line prompt. The f""" prefix enables both string interpolation ({feedback_data}) and line breaks.
• Act as a Product Operations Analyst. — sets the role.
• Task: — specifies what to do.
• Format: — constrains the output shape.
• Data: {feedback_data} — provides the input.
• The comments show the dramatically better output: categorized bullet points instead of a vague summary. For Rae, this is the difference between her bot saying “the customer seems unhappy” and categorizing the ticket as “Technical Issue” or “Feature Request.”

The difference is night and day. The second output is actionable — the guesswork is gone.

The Five Patterns That Work

After thousands of hours of collective testing, five patterns have emerged that resolve almost every communication breakdown. They’re structural components, not one-off tips.

1. Role + Task: Give the model a job.
2. Context + Constraint: Show what matters, and what to avoid.
3. Example-Based: Show, don’t just tell.
4. Chain-of-Thought: Ask the model to think out loud.
5. Structured Output: Force a specific format (like JSON).

Pattern 1: Role + Task — Giving the Model a Job

What’s actually happening when you tell an LLM to “Act as a Senior Data Scientist”? The model doesn’t become that person. It shifts the probabilities of the words it picks, anchoring its perspective and vocabulary.

Without a role, the model defaults to a generic “helpful assistant” tone. Give it one, and the search space narrows to the professional, technical, or creative language appropriate for that persona.

# Let's compare a generic role vs a specific one
raw_data = "Conversion rate dropped by 2% after the update."

# No Role
prompt_1 = f"Explain this: {raw_data}"
# Result: A simple sentence saying the rate went down.

# Specific Role + Task
prompt_2 = f"""
Act as a Senior Growth Lead.
Task: Provide three potential hypotheses for why this conversion drop happened 
and suggest one immediate data deep-dive.
Data: {raw_data}
"""
# Result: A professional breakdown focusing on user funnels and A/B test segments.

This block compares a role-less prompt to one with an explicit role and task.

• raw_data = "Conversion rate dropped by 2% after the update." — a simple string, the data the model will analyze.
• prompt_1 = f"Explain this: {raw_data}" — a minimal f-string that passes the data with no role or structure. The comment shows it produces a bare-bones explanation.
• prompt_2 = f"""...""" — a triple-quoted f-string that adds a role (“Act as a Senior Growth Lead”), a specific task (“Provide three potential hypotheses… and suggest one immediate data deep-dive”), and the data.
• The """ triple quotes — allow the prompt to span multiple lines while still interpolating {raw_data} via the f prefix.
• The comment shows the result: a professional analysis focused on funnels and A/B test segments — the kind of structured output Rae would want when her bot needs to explain why something is happening, not just that it happened.

Pattern 2: Context + Constraint — Showing What Matters

Context gives the model the “why” behind a task. Constraints set the guardrails. Most people struggle here — they provide no context, or far too much.

Ten pages of background? The model gets lost in the noise. You want just enough context that the model understands the stakes, and enough constraints to keep it from rambling.

# Tight Context + Constraints
prompt_context = """
Context: We are a B2B SaaS company launching a new 'Pro' tier. 
Our audience is busy CTOs who value brevity.
Task: Write a 2-sentence announcement email.
Constraint: Do not use the word 'excited' or 'revolutionary'. 
Constraint: Keep the total word count under 30 words.
"""

This block demonstrates tight context and constraints.

• prompt_context = """...""" — a triple-quoted string. Note there’s no f prefix because there’s no variable interpolation needed; this is a static prompt template.
• Context: — explains the situation (B2B SaaS, new Pro tier, busy CTOs who value brevity).
• Task: — specifies the action (2-sentence announcement email).
• Two Constraint: lines — set guardrails: avoid specific words (“excited”, “revolutionary”) and keep under 30 words total.
• The constraints are written as separate lines rather than comma-separated — this is a prompt engineering best practice because the model parses each constraint more reliably when it’s on its own line.
• For Rae’s support bot, this pattern would prevent it from adding fluff like “We’re so excited to help you!” at the start of every response, or from rambling about competitor features when the customer just asked about their own order.

Pattern 3: Example-Based Prompting — Show, Don’t Tell

This is called “Few-Shot Prompting.” It’s perhaps the most useful tool in your kit. Instead of writing a long paragraph explaining the tone you want, give the model two or three examples of what a “good” output looks like.

# Few-Shot Prompting
prompt_examples = """
Task: Convert informal tech support chats into formal ticket summaries.

Example 1:
Input: 'my screen is black lol help'
Output: [Hardware] User reports display failure. Needs troubleshooting.

Example 2:
Input: 'i forgot my password again'
Output: [Access] User requested password reset.

Input: 'the server is down and my boss is mad'
Output:
"""
# The model will now follow the [Category] Description format perfectly.

This block shows few-shot prompting — teaching by example rather than by instruction.

• prompt_examples = """...""" — a triple-quoted static string (no f prefix — no interpolation needed).
• The Task: line — states the goal.
• Two complete Example pairs — each has an Input: (informal chat text) and an Output: (formal ticket summary prefixed with a [Category] tag).
• The final Input: line — provides a new input without a corresponding Output:. This is the actual query the model will respond to, and the blank Output: line signals the model to complete the pattern.
• The model sees the format from the two examples and applies it to the third input. The comment # The model will now follow the [Category] Description format perfectly. confirms that the model learns the output format from demonstration rather than from a description of the rules.
• For Rae’s support bot, this is how she’d teach it to categorize incoming customer messages — by showing it what good categorization looks like rather than trying to describe every possible category in prose.

Pattern 4: Chain-of-Thought — Ask the Model to Think Out Loud

LLMs predict the next word. Ask a complex math question, and the model may give the wrong answer immediately — it hasn’t worked through the steps yet. Prompt it to “think step-by-step,” though, and you force it to generate the intermediate logic. That logic becomes part of its context. The final answer is now much more likely to be correct.

# Chain-of-Thought (CoT)
prompt_cot = """
Task: Calculate the total cost of a subscription.
Details: $10/month, 15% discount for the first 3 months, 5% tax on the final total.
Show your work step-by-step before giving the final number.
"""

This block demonstrates chain-of-thought (CoT) prompting.

• prompt_cot = """...""" — a triple-quoted static string (no interpolation needed).
• Task: — asks for a calculation.
• Details: — provides the numbers ($10/month, 15% discount for 3 months, 5% tax).
• Show your work step-by-step before giving the final number. — the key instruction. It forces the model to generate intermediate calculations before committing to a final answer.
• Without this instruction, the model might predict the final number directly and get it wrong; with it, the intermediate steps become part of the model’s own context window, making the final answer much more likely to be correct.
• For Rae’s support bot, this pattern is essential when a customer asks something like “How much would a 6-month subscription cost with my 20% loyalty discount?” — the bot needs to calculate, not just guess, and CoT forces it to show the math that leads to the right number.

Pattern 5: Structured Output — Tell the Model What Format You Want

When you call an LLM from a Python script, you don’t want conversational text back — you want JSON or CSV. In production, this pattern is critical.

import json

prompt_json = """
Task: Extract entities from the text.
Format: Return ONLY a JSON object with keys: 'person', 'company', 'location'.
Text: 'Elon Musk visited the SpaceX facility in Texas.'
"""

# The model returns: {"person": "Elon Musk", "company": "SpaceX", "location": "Texas"}
# Now you can actually use this in your code!

This block shows structured output prompting for machine-readable responses.

• import json — imports Python’s JSON library, though the actual JSON parsing would happen after the model returns the string, not in this block.
• prompt_json = """...""" — a triple-quoted static string.
• Task: — specifies entity extraction.
• Format: — constrains the output to a JSON object with exactly three keys ('person', 'company', 'location').
• Text: — provides the input sentence.
• The word ONLY in the format instruction — deliberate. Without it, the model might wrap the JSON in conversational padding like “Here’s the extracted information: {…}” which would break json.loads().
• The comment shows the expected model output: a valid JSON object string.
• The second comment — # Now you can actually use this in your code! — highlights why this matters: without structured output, the model might return “The person is Elon Musk, who visited SpaceX in Texas,” which isn’t parseable. With it, Rae can call json.loads(response) and immediately get a dictionary her backend can process — critical for her support bot’s ticket-routing pipeline.

Combining Patterns — The Real Power

You don’t pick just one. You stack them. A production-grade prompt usually layers several of these patterns together.

The Standard Stack:

1. Role (Who are you?)
2. Task (What are you doing?)
3. Context (Why are we doing this?)
4. Examples (What does good look like?)
5. Chain-of-Thought (Work it out first.)
6. Structured Output (Give me the final data.)

Measuring What Actually Works

So what does that mean for Rae? She stops guessing. With her harness in place, she runs each prompt variation against her 20-case test set and watches the score. Change a prompt, and the harness tells her whether it actually got better.

Don’t just eyeball one result. Run the prompt against 10 different inputs and see whether the format breaks or the accuracy improves. In the industry, we call these “Evals.” A simple spreadsheet comparing “Old Prompt Output” vs “New Prompt Output” puts you ahead of 90% of users.

Which Pattern Should Rae Reach For?

Rae’s harness surfaces specific failure modes. Each of the five patterns fixes a different one — and reaching for the wrong pattern wastes tokens without fixing the problem.

Failure Mode the Harness Surfaces	Pattern to Reach For	Why It Works
Bot’s tone is wrong — too casual, too verbose, wrong vocabulary	Role + Task	The role anchors the model’s register and vocabulary to a persona’s language, narrowing the search space
Bot rambles, includes irrelevant info, or violates rules (mentions competitors, over-promises)	Context + Constraint	Context gives the “why”; constraints are guardrails that prevent the model from wandering off-topic
Bot’s format is inconsistent — sometimes a list, sometimes a paragraph, sometimes JSON	Example-Based (Few-Shot)	Examples teach by demonstration; the model pattern-matches the format without you describing it in prose
Bot gets multi-step reasoning wrong — wrong prices, misapplied policies, logic errors	Chain-of-Thought	Forcing intermediate steps lets the model “show its work,” reducing reasoning errors
Bot’s output can’t be parsed by downstream code — conversational padding breaks the pipeline	Structured Output	Explicit format requirements (JSON keys, CSV columns) make the output machine-readable

When NOT to use each pattern:

• Role + Task: Don’t bother if your bot already answers in the right tone — adding a role wastes tokens for no measurable gain. Check your harness: if the relevance score is already high, the role isn’t your problem.
• Context + Constraint: Don’t over-constrain. More than ~3 constraints and the model starts ignoring some of them. Pick the constraints that address the specific failure your harness flagged.
• Example-Based: Don’t use if your examples are low-quality or inconsistent — the model will copy their flaws exactly. Garbage in, garbage out.
• Chain-of-Thought: Don’t use for simple lookup tasks (“What’s our return window?”) — it adds latency and tokens without improving accuracy. Reserve it for multi-step reasoning.
• Structured Output: Don’t use if a human is reading the output directly in a chat interface — conversational text is fine when there’s no downstream parser.

The cost reality: Each pattern adds tokens to your prompt, which means higher API costs and less room in your context window. Stacking all five on a simple task is overkill — and your harness will confirm it by showing diminishing returns. Let the harness tell you which failure you’re fighting, and reach for only the pattern that addresses it.

Common Pitfalls and How to Avoid Them

• Over-specifying: Don’t pile on 50 constraints. The model gets confused and starts ignoring half of them. Pick the top 3.
• Vague Examples: Messy in, messy out. If your examples are sloppy, so is the output.
• Treating it like a human: The model can’t read your mind. If it isn’t in the prompt, the model doesn’t know it.

Putting It Together: A Real-World Example

Here’s a final prompt for a data analyst task: summarizing weekly performance.

final_prompt = """
Role: Senior Data Analyst
Task: Summarize the weekly traffic data provided.
Context: This summary is for the VP of Marketing. Focus on anomalies.

Examples:
- Data: [100, 110, 500, 105] -> Summary: Traffic was stable except for a 400% spike on Wednesday.

Instructions:
1. Analyze the trend.
2. Identify the largest outlier.
3. Output the result in a JSON format with 'trend' and 'outlier' keys.

Data: [200, 210, 190, 800, 205]
"""

This block combines multiple patterns into a single production-grade prompt — the “Standard Stack” in action.

• final_prompt = """...""" — a triple-quoted static string (no f prefix — no interpolation needed). It stacks four patterns:
• Role: — Senior Data Analyst (Role + Task).
• Task: — Summarize weekly traffic data provided.
• Context: — for the VP of Marketing, focus on anomalies (Context + Constraint).
• Examples: — one example showing a data array and the expected summary format (Example-Based).
• Instructions: — three numbered steps including “Output the result in a JSON format” (Structured Output).
• Data: — the actual input array [200, 210, 190, 800, 205].
• The numbered list 1. 2. 3. inside the prompt — gives the model a sequence to follow, which implicitly encourages chain-of-thought reasoning by forcing it to process steps in order before producing the final JSON.
• For Rae, this is the template she’d adapt for her support bot: role (support agent), task (answer the customer), context (product manual + policy docs), examples (good responses from past tickets), and structured output (ticket categorization in JSON) — all measured by her harness to confirm each piece actually improves the score.

Next Steps: When to Go Deeper

Prompt engineering isn’t your only tool.

• If the model needs to know your private company documents, RAG (Retrieval-Augmented Generation) is worth exploring.
• If you need the model to follow a very specific style perfectly every time, Fine-tuning may be the better fit.

Recap and Checklist

• Role: Did you give the model a persona?
• Task: Is the objective clear and verb-based?
• Examples: Did you show it 2-3 “gold standard” outputs?
• Steps: Did you ask it to think step-by-step?
• Format: Did you specify JSON or a list?

Start using these patterns today and you’ll spend less time correcting the model and more time directing it.

Rae applies these patterns systematically, running each iteration through her harness and watching the scores climb. After a few weeks, though, her scores plateau. The bot nails tone, format, and reasoning — but it still can’t answer questions about her company’s specific return policy or her product’s three-tier warranty. No prompt pattern can inject knowledge the model doesn’t have. In the next article, we’ll map out her three real options — prompt engineering, in-context learning, and fine-tuning — and when each one stops being enough.

Check Your Understanding

Questions below move from simple recall up to open-ended design, roughly following Bloom’s Taxonomy.

Remember What are the five prompt patterns the article names, and what does each one address?

Understand In your own words, explain why Chain-of-Thought prompting improves accuracy on math questions, using the article’s explanation of how LLMs predict the next word.

Apply Using the article’s “Standard Stack” order (Role → Task → Context → Examples → Chain-of-Thought → Structured Output), where would you insert a new constraint like “keep the response under 100 words” — as part of Task, Context, or a separate step?

Analyze The article’s Pitfall list warns against “Over-specifying” (more than ~3 constraints causes the model to ignore half of them). Walk through why stacking all six patterns from the “Standard Stack” in a single production prompt doesn’t count as over-specifying, while listing 10 unrelated formatting rules would.

Evaluate The article recommends running a new prompt “against 10 different inputs” and comparing “Old Prompt Output” vs “New Prompt Output” in a spreadsheet as an eval strategy. Critique this as a complete evaluation approach: what does eyeballing 10 outputs in a spreadsheet fail to catch that a systematic harness (with defined scoring, not just visual comparison) would catch?

Create Design a production-grade prompt (following the article’s “Standard Stack”) for a new task: an internal tool that reads a customer’s support ticket and drafts a first-response email. Write out Role, Task, Context, one Example, and the Structured Output format you’d specify, and explain why you included each piece.

---

Related articles

• Building a Simple LLM Evaluation Harness in Python
• Fine-Tuning vs. Prompt Engineering vs. In-Context Learning
• Evaluating LLM Output Beyond “It Looks Right”
• Fine-Tuning vs. RAG: How to Actually Decide

References & Further reading

• Wei, J., Wang, X., Schuurmans, D., et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS 2022. arxiv.org/abs/2201.11903
• OpenAI. Prompt Engineering Guide. platform.openai.com/docs/guides/prompt-engineering

---