Last time, Rae applied five prompt engineering patterns—Role + Task, Context + Constraint, Example-Based, Chain-of-Thought, and Structured Output—running each iteration through her evaluation harness and watching the scores climb. But after weeks of iteration, her scores plateaued. The bot nailed tone, format, and reasoning. It still couldn’t answer questions about her company’s return policy or her product’s three-tier warranty, because no prompt pattern could inject knowledge the model didn’t have. Some failures needed the model to know more—not just be instructed better.

Rae’s support bot is like a genius intern she just hired. It knows everything about history, science, and literature, but it has no idea how her company works—what the return policy says, what the three-tier warranty covers, how to route a refund escalation. She wouldn’t just fire it and hire a different model, right? She’d guide it.

Large Language Models (LLMs) like GPT-4 or Llama-3 are exactly like that intern. They arrive “pre-trained” on the whole internet, but they aren’t task-specific. To get them doing what you want, you have three main levers: Prompt Engineering, In-Context Learning, and Fine-Tuning.

So:

• Prompt Engineering is giving clear instructions.
• In-Context Learning is showing a few examples of the work.
• Fine-Tuning is sending them to a specialized six-month training camp to change how their brain works.

Each lever trades off speed, cost, and permanence. Here’s what happens when we try a simple task—classifying a customer review—using just a basic instruction.

import openai

# A simple setup to see how we steer the model
def get_llm_response(prompt):
    # Imagine this calls your favorite LLM API
    # For this example, we'll simulate the logic
    if "Instruction:" in prompt and "Example:" not in prompt:
        return "Response: Positive (based on instructions)"
    elif "Example:" in prompt:
        return "Response: Positive (based on patterns)"
    return "Response: I'm not sure what to do."

# Level 1: Basic Prompt Engineering
instruction_prompt = """
Instruction: Classify the sentiment of the following review as Positive or Negative.
Review: The battery life is amazing and the screen is crisp.
"""
print(get_llm_response(instruction_prompt))

This block simulates an LLM API call to demonstrate the three steering levers at a high level — no real API key is needed since the logic is mocked with string matching.

• import openai — imports the OpenAI Python library, though it’s not actually used here since the function below is a mock.
• def get_llm_response(prompt): — defines a function that takes a prompt string and returns a simulated response based on simple string matching.
• if "Instruction:" in prompt and "Example:" not in prompt: — checks whether the prompt contains the word “Instruction:” but NOT “Example:” — this branch represents pure prompt engineering (instructions only, no examples).
• elif "Example:" in prompt: — checks whether the prompt contains examples — this branch represents in-context learning.
• return "Response: I'm not sure what to do." — the fallback for prompts that have neither instructions nor examples.
• instruction_prompt — a triple-quoted string giving the model a clear instruction: classify a review’s sentiment as Positive or Negative.
• print(get_llm_response(instruction_prompt)) — calls the mock function with this prompt; since it contains “Instruction:” but not “Example:”, the function returns "Response: Positive (based on instructions)" (confirmed by running this block).

For Rae, this is Level 1: her support bot gets a basic instruction and responds, but without examples or fine-tuning, it might miss nuances like sarcasm or complex multi-part customer complaints.

the model above is just following a rule. It’s fast and free, but it’ll struggle when the reviews get sarcastic or complex.

Prompt Engineering: Talking to the Model Better

Prompt engineering is the art of writing better instructions. You aren’t changing the model’s “brain” (the weights); you’re just being more clever about what you ask.

Here’s the thing: the model isn’t a mind reader. Vague prompt in, vague answer out. Good prompts lean on clarity, structure, and Chain-of-Thought — that last one means asking the model to think step-by-step.

So prompt engineering is a pure input-side intervention. It’s fast (milliseconds) and reversible. Don’t like the answer? Just change the words.

# Before: Vague
vague_prompt = "Review this: The food was okay, I guess."

# After: Structured with Role-Play and Chain-of-Thought
structured_prompt = """
You are a senior customer satisfaction analyst.
Task: Analyze the sentiment of the review below.
Process: 
1. Identify the key subjects mentioned.
2. Determine if the tone is sarcastic.
3. Provide a final label: Positive, Negative, or Neutral.

Review: The food was okay, I guess.
"""

# The output for the second prompt will be much more reliable 
# because we gave the model a 'path' to follow.

This block contrasts a vague prompt with a structured one to show how prompt engineering improves output quality.

• vague_prompt = "Review this: The food was okay, I guess." — a minimal one-line prompt with no role, no structure, and no process — the model has to guess what kind of analysis is wanted.
• structured_prompt = """...""" — a triple-quoted string that adds three elements:
  • You are a senior customer satisfaction analyst. — sets a role, anchoring the model’s vocabulary and perspective.
  • Task: Analyze the sentiment of the review below. — specifies the objective.
  • Process: with three numbered steps (identify key subjects, determine if the tone is sarcastic, provide a final label) — implements Chain-of-Thought, forcing the model to generate intermediate reasoning before committing to a label.
• # The output for the second prompt will be much more reliable because we gave the model a 'path' to follow. — highlights the key insight: the structured prompt gives the model a reasoning path, so it’s more likely to correctly identify that “okay, I guess” is ambiguous or neutral rather than confidently labeling it Positive.

For Rae’s support bot, this is the difference between “classify this ticket” (vague) and “act as a support analyst, analyze the issue step-by-step, then categorize it” (structured) — the same patterns she applied in the previous article, now serving as the baseline she measures against before reaching for a more expensive lever.

In-Context Learning: Teaching by Example (Without Retraining)

In-context learning is prompt engineering’s smarter sibling. Rather than just telling the model what to do, you show it. The term for this is “few-shot prompting.”

Here’s the thing: the model isn’t actually “learning” in the sense of updating its memory. It’s just using the examples in its current “view” (the context window) to spot a pattern. Think of showing a child three pictures of a dog before asking them to identify a fourth.

few_shot_prompt = """
Classify the sentiment of these reviews.

Review: The movie was a total waste of time.
Label: Negative

Review: I've never been happier with a purchase!
Label: Positive

Review: It was fine, nothing special.
Label: Neutral

Review: The wait time was long, but the steak was worth it.
Label:
"""
# The model sees the pattern (Review -> Label) and completes it.

This block demonstrates in-context learning (few-shot prompting) — teaching by example rather than by instruction.

• few_shot_prompt = """...""" — a triple-quoted static string (no f prefix — no variable interpolation needed).
• The task line — Classify the sentiment of these reviews. — sets up the pattern the model is meant to follow.
• Three complete Review: + Label: pairs — a Negative example (“total waste of time”), a Positive example (“never been happier”), and a Neutral example (“fine, nothing special”).
• The fourth Review: line — “The wait time was long, but the steak was worth it.” — is followed by Label: with no value after it. This blank label is the model’s cue to complete the pattern: it sees the Review → Label mapping from the three examples and applies it to the new mixed-sentiment review.
• # The model sees the pattern (Review -> Label) and completes it. — confirms that the model learns the format from demonstration, not from a description of the rules.

For Rae, this is how she’d teach her support bot to categorize incoming tickets — show it three labeled examples, then give it a new ticket to classify using the same pattern. The key distinction from fine-tuning: the model’s weights haven’t changed. Remove these examples from the prompt and the model forgets the pattern entirely.

the accuracy goes up because the model doesn’t have to guess your preferred format. So there’s a ceiling—you can only fit so many examples before hitting the “context window” limit (the model’s short-term memory).

Fine-Tuning: Permanently Changing the Model

Fine-tuning is the only approach that actually changes the model’s weights. You’re retraining the model on your specific data. Once you fine-tune, the model “knows” your task inherently.

It’s slower and more expensive — you need a GPU and a labeled dataset. But for specialized tasks like writing legal briefs in a specific firm’s style, it’s often the only way.

# Anatomy of a fine-tuning preparation
dataset = [
    {"prompt": "Review: The UI is clunky. ->", "completion": " Negative"},
    {"prompt": "Review: Best app ever! ->", "completion": " Positive"}
    # ... imagine 500 more examples here
]

print(f"Prepared {len(dataset)} examples for weight updates.")
# After training, the model doesn't need instructions anymore. 
# It just 'knows' that '->' means 'classify this'.

This block shows the structure of a fine-tuning dataset — the data you’d prepare to actually change the model’s weights.

• dataset = [...] — a list of dictionaries, each with a "prompt" key (the input) and a "completion" key (the expected output).
• "Review: The UI is clunky. ->" — uses an arrow (->) as a delimiter between the review text and the expected classification; the completion " Negative" is the label the model should produce after seeing that arrow.
• # ... imagine 500 more examples here — signals this is a toy dataset; real fine-tuning requires hundreds or thousands of labeled examples.
• print(f"Prepared {len(dataset)} examples for weight updates.") — uses an f-string to report the dataset size; len(dataset) returns 2 for this toy example (confirmed by running this block).
• # After training, the model doesn't need instructions anymore. It just 'knows' that '->' means 'classify this'. — highlights the fundamental difference from prompt engineering and in-context learning: after fine-tuning, the model has internalized the task. The -> convention is baked into the weights, not the prompt.

For Rae, this means if she fine-tunes on 500+ labeled support tickets, the bot would automatically categorize new tickets without needing elaborate prompts or few-shot examples in every call — it “knows” the task from training, not from instructions it reads at inference time.

Head-to-Head: Cost, Speed, and Flexibility

So what does that mean in practice? Here’s how the three approaches stack up.

Feature	Prompt Engineering	In-Context Learning	Fine-Tuning
Upfront Cost	$0	$0	High (Compute + Data)
Setup Speed	Seconds	Minutes	Days/Weeks
Accuracy	Good	Better	Best (for specific tasks)
Flexibility	High (just change text)	High	Low (must retrain)

For a simple spam detector, prompt engineering is plenty. In-context learning shines when you have a few edge cases to handle. Fine-tuning only makes sense when the stakes are high — a medical diagnostic tool requiring 99.9% accuracy, for instance.

A Real Example: Customer Support Ticket Classification

Here’s how the three handle a support ticket: “My order #123 hasn’t arrived, and I’m angry!”

1. Prompting: “Classify this as Shipping, Billing, or Technical.” -> Result: Shipping (Usually works).
2. In-Context: Show 5 examples of angry shipping complaints. -> Result: Shipping - Urgent (The model catches the ‘angry’ nuance from your examples).
3. Fine-Tuning: Train on 1,000 tickets labeled by your actual staff. -> Result: Shipping - Delayed - Refund Requested (The model learns your internal sub-categories — the ones a prompt can’t anticipate).

The Hybrid Approach: Combining All Three

In practice, we rarely pick just one. The pros layer these techniques.

Fine-tune a model on your industry’s jargon, then give it a specific persona through a well-structured prompt. Add 2-3 examples of the current task on top of that. This “Triple Threat” approach is how production-grade AI systems actually work.

When to Fine-Tune (And When NOT To)

Here’s the catch: fine-tuning can make a model worse if you aren’t careful.

Fine-tune if:

• You have 500+ high-quality labeled examples.
• The model needs to learn a very specific format (like JSON with custom keys).
• You need to cut latency by shortening your prompts.

Don’t fine-tune if:

• Your data changes every week.
• You just want to give the model new facts (use RAG instead).
• You haven’t tried a good prompt yet.

Which Lever Should Rae Pull?

Rae has three levers, and each one trades off differently on speed, cost, and permanence. Here’s the comparison that matters for her startup:

Lever	Speed to Deploy	Cost	Permanence	What It Actually Changes
Prompt Engineering	Seconds (just edit text)	$0 — only API token cost	Zero — disappears when you change the prompt	Nothing in the model; just the input text
In-Context Learning	Minutes (add 3-5 examples to the prompt)	$0 — slightly more tokens per call	Zero — examples vanish when you remove them from the prompt	The model’s current “view” — it pattern-matches from what it sees
Fine-Tuning	Days to weeks (prepare data, train, evaluate)	High — GPU compute + labeling labor	Permanent — the model’s weights are changed until you retrain	The model’s actual “brain” — it internalizes the task

The decision guide for Rae’s startup:

1. Start with Prompt Engineering. It’s free, reversible, and solves 70% of problems. If Rae’s bot gives a wrong answer because the prompt was vague, no amount of fine-tuning will fix that — better instructions will. She should exhaust this lever first, using her evaluation harness to confirm the improvement before spending money.

2. Add In-Context Learning when the format or pattern is inconsistent. If the bot’s tone is right but its output format varies — sometimes JSON, sometimes prose — 3-5 examples in the prompt will stabilize it. This costs a few extra tokens per call but zero upfront investment. If adding examples to the prompt moves the harness score, she doesn’t need fine-tuning yet.

3. Reach for Fine-Tuning only when the first two plateau — and you have the data. Rae needs 500+ high-quality labeled examples, a real compute budget, and a task that won’t change next week. Fine-tuning is permanent: once she trains the model on her company’s support tickets, it “knows” that task — but if her ticket categories change, she has to retrain. For a startup with limited budget, LoRA (Parameter-Efficient Fine-Tuning) is the realistic path: it trains ~1% of the weights, costs a fraction of full fine-tuning, and gets ~98% of the performance.

When NOT to move up the ladder:

• Don’t fine-tune if your data changes frequently — you’ll be retraining constantly.
• Don’t fine-tune to inject knowledge (like your product manual) — that’s what RAG is for. Fine-tuning is for style and task logic, not facts.
• Don’t skip the harness. The whole point of having evaluation scores is to know when prompt engineering has actually plateaued and it’s time to spend money. Without those numbers, you’re guessing.

Parameter-Efficient Fine-Tuning: The Middle Ground

Full fine-tuning updates all billions of parameters — overkill for most people. LoRA (Low-Rank Adaptation) is the standard alternative.

Think of it as adding a small plugin to the model. You only train about 1% of the weights. It’s 10x faster, needs way less memory, and gets you 98% of the way to a full fine-tune.

Putting It All Together: A Decision Framework

Now let’s run this through a decision tree:

def choose_strategy(has_data, task_complexity, budget):
    if not has_data:
        return "Prompt Engineering"
    if has_data and task_complexity == "Low":
        return "In-Context Learning"
    if has_data and budget == "High":
        return "Fine-Tuning (LoRA)"
    return "Start with Prompting, then iterate."

print(f"Strategy: {choose_strategy(True, 'High', 'High')}")

This block implements a simple decision function for choosing between the three levers.

• def choose_strategy(has_data, task_complexity, budget): — defines a function with three parameters — has_data (boolean), task_complexity (string), and budget (string).
• if not has_data: — returns "Prompt Engineering" — if you don’t have labeled data, you can’t do in-context learning or fine-tuning, so you’re stuck with instructions.
• if has_data and task_complexity == "Low": — returns "In-Context Learning" — if the task is simple and you have data, showing a few examples in the prompt is enough.
• if has_data and budget == "High": — returns "Fine-Tuning (LoRA)" — if you have data AND the budget for compute, fine-tuning via LoRA is the most permanent solution.
• return "Start with Prompting, then iterate." — the fallback for cases that don’t cleanly match any branch.
• print(f"Strategy: {choose_strategy(True, 'High', 'High')}") — calls the function with has_data=True, task_complexity='High', and budget='High'. Since not has_data is False, it skips the first branch; has_data and task_complexity == "Low" is False (complexity is “High”), so it skips the second; has_data and budget == "High" is True, so it returns "Fine-Tuning (LoRA)" (confirmed by running this block: Strategy: Fine-Tuning (LoRA)).

For Rae, this encodes the decision tree she’s facing: her support bot has labeled data (past tickets), high task complexity (product-specific questions), and a startup budget she’s willing to invest — the function tells her LoRA fine-tuning is the next step. The real question she still has to answer: is her problem actually a fine-tuning problem, or is it a RAG problem? That’s the next article.

Common Pitfalls and How to Avoid Them

1. Over-tuning: Fine-tuning on 20 examples. The model memorizes them and flops on everything else. That’s overfitting.
2. Ignoring the Baseline: People jump straight to fine-tuning before checking whether a plain prompt does the job. Measure your baseline first.
3. Prompt Drift: A prompt that works on the base model can break on your fine-tuned version. Re-test everything.

What’s Next: Fine-Tuning vs. RAG

There’s a fourth player in this game: RAG. If your goal is to give the model access to your company’s private PDFs or the latest news, fine-tuning is the wrong tool. Fine-tuning is for style and task logic; RAG is for knowledge.

For Rae, the decision narrows to two real contenders for her specific problem: fine-tune on her company’s data—every past support ticket, every resolved refund, every warranty claim—or push further into RAG, pulling relevant passages from her product manual into the context at query time. One changes the model’s brain. The other changes what it can see. In the next article, we’ll put them head-to-head and figure out which one Rae should pick.

Key Takeaways

• Prompt Engineering is your first stop—it’s cheap and fast.
• In-Context Learning adds examples to guide the model’s pattern matching.
• Fine-Tuning changes the model’s brain but requires data and compute.
• LoRA is the smart way to fine-tune without breaking the bank.
• Always measure before you move to a more expensive method.

Check Your Understanding

Questions below move from simple recall up to open-ended design, roughly following Bloom’s Taxonomy.

Remember What is the key difference between In-Context Learning and Fine-Tuning in terms of whether the model’s weights actually change?

Understand In your own words, explain why the article says fine-tuning is the wrong tool for giving a model access to your company’s latest PDFs, and RAG is the right one — what’s the difference between “style/task logic” and “knowledge”?

Apply Using the article’s choose_strategy decision function, what strategy would it return for a team with has_data=True, task_complexity="Low", and budget="High"? Trace through the if/elif logic to explain why.

Analyze The article’s Pitfall #1 warns that fine-tuning on only 20 examples causes overfitting (“memorize them and fail on everything else”). Walk through why In-Context Learning doesn’t have this same failure mode even though it also relies on a small number of examples (3-5 shown in the prompt).

Evaluate The article’s decision table rates Fine-Tuning’s “Flexibility” as “Low (must retrain)” but “Accuracy” as “Best.” Critique treating this as a simple accuracy-vs-flexibility tradeoff for a company whose product requirements change monthly: what ongoing cost does the table not capture that would matter more than the one-time setup cost?

Create Design a “Triple Threat” hybrid pipeline (the article’s fine-tune + prompt + examples layering) for a new use case: an internal tool that drafts replies to vendor contract negotiation emails in a specific legal team’s tone. Specify what you’d fine-tune on, what persona/instructions you’d put in the prompt, and what few-shot examples you’d include, and justify why this task needs all three layers rather than just one.

---

Related articles

• Why Your Prompts Fail (And What That Tells Us)
• Fine-Tuning vs. RAG: How to Actually Decide
• Building a Simple LLM Evaluation Harness in Python

References & Further reading

• Brown, T. B., Mann, B., Ryder, N., et al. (2020). Language Models are Few-Shot Learners. NeurIPS 2020. arxiv.org/abs/2005.14165 — the GPT-3 paper that introduced and formalized in-context learning.
• OpenAI. Fine-tuning documentation. platform.openai.com/docs/guides/fine-tuning

---