Lesson 3.3: Few-Shot Learning

Duration: 60 minutes

Learning Objectives

By the end of this lesson, you will be able to:

• Understand what few-shot learning is and when to use it
• Write effective few-shot prompts with well-chosen examples
• Select and structure examples for maximum effectiveness
• Handle edge cases and improve consistency
• Compare zero-shot, one-shot, and few-shot approaches

Introduction

Sometimes telling an AI what to do is not enough - you need to show it. Few-shot learning is the technique of teaching a model by providing examples in your prompt. Instead of writing complex instructions, you demonstrate the pattern you want through input-output pairs.

This is one of the most powerful techniques in prompt engineering, and it works because LLMs are excellent pattern matchers. Give them a few examples, and they will extrapolate the pattern to new inputs.

Understanding Few-Shot Learning

The Concept

Few-shot learning means providing a "few" examples (typically 2-5) to demonstrate a task:

┌─────────────────────────────────────────────────────────┐
│                Few-Shot Learning Flow                    │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  Example 1:  Input → Output                             │
│  Example 2:  Input → Output                             │
│  Example 3:  Input → Output                             │
│                                                         │
│  ─────────────────────────────                          │
│                                                         │
│  New Input:  ?                                          │
│                                                         │
│  Model applies the learned pattern                      │
│                                                         │
│  New Output: (follows the pattern)                      │
│                                                         │
└─────────────────────────────────────────────────────────┘

Terminology

• Zero-shot: No examples provided, just instructions
• One-shot: One example provided
• Few-shot: Multiple examples provided (typically 2-5)
• Many-shot: Large number of examples (can consume context window)

Basic Few-Shot Example

Let us classify customer feedback sentiment:

import Anthropic from '@anthropic-ai/sdk';

const anthropic = new Anthropic();

const prompt = `
Classify the sentiment of customer feedback as positive, negative, or neutral.

Examples:

Feedback: "The product arrived quickly and works perfectly. Love it!"
Sentiment: positive

Feedback: "It's okay, nothing special but does what it's supposed to."
Sentiment: neutral

Feedback: "Terrible experience. Product broke after one week and support was unhelpful."
Sentiment: negative

Feedback: "The shipping was delayed but the product quality is excellent."
Sentiment:`;

const response = await anthropic.messages.create({
  model: 'claude-sonnet-4-20250514',
  max_tokens: 50,
  messages: [{ role: 'user', content: prompt }],
});

console.log(response.content[0].text);
// Output: "neutral" or "positive" (mixed feedback)

The model learns from the examples:

• Positive feedback contains praise and satisfaction
• Neutral feedback is balanced or indifferent
• Negative feedback contains complaints and frustration

When to Use Few-Shot Learning

Good Use Cases

1. Format specification: When you need a specific output format

const extractionPrompt = `
Extract product information from descriptions.

Example 1:
Description: "Samsung Galaxy S24 Ultra, 256GB storage, Titanium Black color, $1199"
Output: { "brand": "Samsung", "model": "Galaxy S24 Ultra", "storage": "256GB", "color": "Titanium Black", "price": 1199 }

Example 2:
Description: "Apple iPhone 15 Pro Max with 512GB in Natural Titanium for $1399"
Output: { "brand": "Apple", "model": "iPhone 15 Pro Max", "storage": "512GB", "color": "Natural Titanium", "price": 1399 }

Description: "${userInput}"
Output:`;

2. Classification tasks: When labels are not intuitive

const classificationPrompt = `
Classify support tickets by priority.

Ticket: "My account was hacked and someone is making purchases"
Priority: P0-Critical

Ticket: "The website is loading slowly today"
Priority: P2-Medium

Ticket: "How do I change my email address?"
Priority: P3-Low

Ticket: "Payment failed but money was charged twice"
Priority: P1-High

Ticket: "${newTicket}"
Priority:`;

3. Style matching: When you need a specific writing style

const stylePrompt = `
Convert formal text to casual social media posts.

Formal: "We are pleased to announce the launch of our new product line."
Casual: "Big news! Our new products just dropped and we're SO excited to share them with you!

Formal: "Customers are advised that maintenance will occur on Saturday."
Casual: "Heads up! We're doing some behind-the-scenes work this Saturday. Things might be a bit wonky.

Formal: "${formalText}"
Casual:`;

4. Translation patterns: Not just languages, but formats

const sqlPrompt = `
Convert natural language to SQL queries.

Question: "Show all users who signed up in 2024"
SQL: SELECT * FROM users WHERE YEAR(created_at) = 2024;

Question: "Count orders by status"
SQL: SELECT status, COUNT(*) as count FROM orders GROUP BY status;

Question: "Find products with price over $100 sorted by name"
SQL: SELECT * FROM products WHERE price > 100 ORDER BY name ASC;

Question: "${userQuestion}"
SQL:`;

When Zero-Shot is Better

Few-shot is not always necessary. Use zero-shot when:

• The task is straightforward and well-defined
• You want to save tokens (and money)
• Examples might bias the model toward limited patterns

Zero-shot example:

const zeroShotPrompt = `
Summarize the following article in 2-3 sentences.
Focus on the main points and key takeaways.

Article:
${articleText}

Summary:`;

Crafting Effective Examples

Principle 1: Diversity

Choose examples that cover different scenarios:

Bad: Similar examples

Example 1: "I love this product!" → positive
Example 2: "This is amazing!" → positive
Example 3: "Best purchase ever!" → positive

Good: Diverse examples

Example 1: "I love this product!" → positive
Example 2: "It's okay, nothing special." → neutral
Example 3: "Completely disappointed, waste of money." → negative
Example 4: "Good quality but shipping was slow." → mixed

Principle 2: Representativeness

Examples should match real-world inputs your system will receive:

// If your users write informal messages, use informal examples
const realisticExamples = `
Example 1:
User: "hey how do i reset my pw"
Intent: password_reset

Example 2:
User: "ur app keeps crashing on my iphone!!"
Intent: bug_report

Example 3:
User: "can someone help me"
Intent: general_support
`;

// Not overly polished examples like:
const unrealisticExamples = `
Example 1:
User: "Hello, I would like to request a password reset for my account."
Intent: password_reset
`;

Principle 3: Clarity

Make the input-output relationship obvious:

Bad: Ambiguous

Input: "apple"
Category: fruit

Input: "Apple"
Category: company

Good: Clear context

Product listing: "Fresh organic apples, 5lb bag"
Category: groceries/fruit

Product listing: "Apple MacBook Pro 14-inch M3"
Category: electronics/computers

Principle 4: Consistent Format

Use the same structure for all examples:

// Consistent structure
const consistentPrompt = `
Convert these notes to action items.

Notes: "Need to call John about the project deadline. Also remember to book flights for next week's conference."
Action items:
1. Call John about project deadline
2. Book conference flights for next week

Notes: "Review the PR from Sarah. Update documentation with new API endpoints."
Action items:
1. Review Sarah's PR
2. Update documentation with new API endpoints

Notes: "${userNotes}"
Action items:
`;

Advanced Few-Shot Techniques

Negative Examples

Show what NOT to do:

const withNegativeExamples = `
Generate professional email subject lines.

Email content: "Meeting rescheduled from Monday to Wednesday"
Good subject: "Meeting Rescheduled: Now Wednesday at 10am"
Bad subject: "URGENT!!! MEETING CHANGED!!!"

Email content: "Quarterly report is ready for review"
Good subject: "Q3 Report Ready for Review"
Bad subject: "report done"

Email content: "${emailContent}"
Subject:`;

Edge Case Examples

Include tricky cases to handle ambiguity:

const withEdgeCases = `
Extract dates from text. Return "none" if no date is mentioned.

Text: "Let's meet on March 15th, 2024"
Date: 2024-03-15

Text: "The project started last Monday"
Date: relative (last Monday)

Text: "I enjoyed the conference"
Date: none

Text: "Sometime next week works for me"
Date: relative (next week)

Text: "${inputText}"
Date:`;

Multi-Step Examples

For complex tasks, show the reasoning process:

const multiStepPrompt = `
Analyze code for potential improvements.

Code:
\`\`\`typescript
function getData(id) {
  const data = fetch('/api/' + id);
  return data;
}
\`\`\`

Analysis:
1. Issue: No TypeScript types
   Severity: Medium
   Fix: Add parameter and return types

2. Issue: String concatenation in URL (potential injection)
   Severity: High
   Fix: Use template literals or URL builder

3. Issue: Missing await for async fetch
   Severity: Critical
   Fix: Add async/await

Improved code:
\`\`\`typescript
async function getData(id: string): Promise<Response> {
  const data = await fetch(\`/api/\${encodeURIComponent(id)}\`);
  return data;
}
\`\`\`

---

Code:
\`\`\`typescript
${userCode}
\`\`\`

Analysis:`;

Implementing Few-Shot in Code

Reusable Few-Shot Function

import OpenAI from 'openai';

interface Example {
  input: string;
  output: string;
}

interface FewShotConfig {
  taskDescription: string;
  examples: Example[];
  inputLabel?: string;
  outputLabel?: string;
}

function buildFewShotPrompt(config: FewShotConfig, newInput: string): string {
  const inputLabel = config.inputLabel || 'Input';
  const outputLabel = config.outputLabel || 'Output';

  let prompt = config.taskDescription + '\n\n';

  for (const example of config.examples) {
    prompt += `${inputLabel}: ${example.input}\n`;
    prompt += `${outputLabel}: ${example.output}\n\n`;
  }

  prompt += `${inputLabel}: ${newInput}\n`;
  prompt += `${outputLabel}:`;

  return prompt;
}

// Usage
const sentimentConfig: FewShotConfig = {
  taskDescription: 'Classify the sentiment of the text as positive, negative, or neutral.',
  examples: [
    { input: 'This is the best day ever!', output: 'positive' },
    { input: "I'm so frustrated with this service.", output: 'negative' },
    { input: 'The meeting is at 3pm.', output: 'neutral' },
  ],
  inputLabel: 'Text',
  outputLabel: 'Sentiment',
};

const prompt = buildFewShotPrompt(sentimentConfig, "I can't believe how great this turned out!");

Dynamic Example Selection

Select relevant examples based on the input:

interface LabeledExample {
  input: string;
  output: string;
  category: string;
  keywords: string[];
}

const examplePool: LabeledExample[] = [
  {
    input: 'How do I return an item?',
    output: 'returns',
    category: 'support',
    keywords: ['return', 'refund', 'send back'],
  },
  {
    input: "My order hasn't arrived yet",
    output: 'shipping',
    category: 'support',
    keywords: ['order', 'delivery', 'shipping', 'arrived'],
  },
  {
    input: 'Can I change my password?',
    output: 'account',
    category: 'support',
    keywords: ['password', 'login', 'account', 'email'],
  },
  // ... more examples
];

function selectRelevantExamples(
  input: string,
  pool: LabeledExample[],
  maxExamples: number = 3
): LabeledExample[] {
  const inputLower = input.toLowerCase();

  // Score examples by keyword matches
  const scored = pool.map((example) => {
    const score = example.keywords.filter((kw) => inputLower.includes(kw)).length;
    return { example, score };
  });

  // Sort by score, take top N
  return scored
    .sort((a, b) => b.score - a.score)
    .slice(0, maxExamples)
    .map((s) => s.example);
}

// Usage
const userQuery = 'I want to return my damaged package';
const relevantExamples = selectRelevantExamples(userQuery, examplePool);

Caching Examples for Performance

import OpenAI from 'openai';

const openai = new OpenAI();

class FewShotClassifier {
  private examples: Example[];
  private cachedPromptPrefix: string;

  constructor(
    private taskDescription: string,
    examples: Example[]
  ) {
    this.examples = examples;
    this.cachedPromptPrefix = this.buildPromptPrefix();
  }

  private buildPromptPrefix(): string {
    let prompt = this.taskDescription + '\n\n';
    for (const example of this.examples) {
      prompt += `Input: ${example.input}\nOutput: ${example.output}\n\n`;
    }
    return prompt;
  }

  async classify(input: string): Promise<string> {
    const fullPrompt = this.cachedPromptPrefix + `Input: ${input}\nOutput:`;

    const response = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [{ role: 'user', content: fullPrompt }],
      max_tokens: 50,
      temperature: 0,
    });

    return response.choices[0].message.content?.trim() || '';
  }

  // Add examples dynamically
  addExample(example: Example): void {
    this.examples.push(example);
    this.cachedPromptPrefix = this.buildPromptPrefix();
  }
}

// Usage
const classifier = new FewShotClassifier('Classify the programming language of the code snippet.', [
  { input: "console.log('hello')", output: 'JavaScript' },
  { input: "print('hello')", output: 'Python' },
  { input: 'fmt.Println("hello")', output: 'Go' },
]);

const language = await classifier.classify('System.out.println("hello");');
console.log(language); // "Java"

Comparing Approaches

Zero-Shot vs Few-Shot: When to Choose

// Zero-shot: Clear, well-defined task
const zeroShotSummarize = `
Summarize this article in 3 bullet points:
${article}
`;

// Few-shot: Subjective or format-specific task
const fewShotSummarize = `
Summarize articles in a specific style with emoji headers.

Article: "Apple announced new MacBooks with M3 chips offering better performance."
Summary:
💻 Apple launches new MacBook lineup
⚡ M3 chips bring significant performance gains
🔋 Expected battery life improvements

Article: "${article}"
Summary:
`;

Number of Examples: Finding the Sweet Spot

// One-shot: Simple patterns
const oneShot = `
Convert temperatures. Example: "32F" → "0C"
Convert: "${temp}"
`;

// Few-shot (3-5): Most classification tasks
const fewShot = `
Classify intent (3 examples covering main categories)...
`;

// Many-shot: Complex, nuanced tasks (but watch token usage!)
const manyShot = `
Classify legal document types (10+ examples for legal nuance)...
`;

Performance Comparison

Troubleshooting Few-Shot Prompts

Problem: Model Copies Examples Too Literally

Symptom: Output matches example format but copies content

Solution: Add diversity and explicit instructions

// Add this instruction
const prompt = `
Classify sentiment. Generate a NEW classification for each input.
Do not copy the example outputs verbatim.

Examples:
${examples}

New input: ${input}
`;

Problem: Inconsistent Output Format

Symptom: Sometimes JSON, sometimes plain text

Solution: Be explicit about format in examples and add instruction

const prompt = `
Extract entities as JSON. ALWAYS return valid JSON only.

Text: "John works at Google"
Output: {"person": "John", "company": "Google"}

Text: "The meeting is in New York"
Output: {"person": null, "location": "New York"}

Text: "${input}"
Output:`;

Problem: Model Ignores Some Examples

Symptom: Certain patterns not learned

Solution: Put important examples last (recency bias)

// Order examples with the most important patterns last
const examples = [
  simpleCase, // First
  anotherSimple,
  edgeCase, // Important: near end
  criticalPattern, // Most important: last
];

Exercises

Exercise 1: Build a Few-Shot Classifier

Create a few-shot prompt to classify emails into categories: work, personal, spam, newsletter.

const emailClassifier = `
Classify emails into categories: work, personal, spam, newsletter.

Email: "Hi team, please review the Q3 report by Friday. Thanks, Manager"
Category: work

Email: "Hey! Are we still on for dinner Saturday? Let me know!"
Category: personal

Email: "CONGRATULATIONS! You've won $1,000,000! Click here to claim!"
Category: spam

Email: "Your weekly digest from TechNews - Top stories this week..."
Category: newsletter

Email: "Meeting rescheduled to 3pm. Conference room B."
Category: work

Email: "${emailContent}"
Category:`;

Exercise 2: Handle Edge Cases

The following few-shot prompt fails on edge cases. Improve it:

Text: "happy" → positive
Text: "sad" → negative
Text: "${input}" →

const improvedPrompt = `
Classify text sentiment as positive, negative, neutral, or mixed.

Text: "I love this!"
Sentiment: positive

Text: "This is terrible."
Sentiment: negative

Text: "The meeting is at 3pm."
Sentiment: neutral

Text: "Good product but shipping was awful."
Sentiment: mixed

Text: "meh"
Sentiment: neutral

Text: ""
Sentiment: neutral

Text: "${input}"
Sentiment:`;

Exercise 3: Create a Translation Prompt

Build a few-shot prompt that converts English cooking instructions to metric measurements.

const metricConverter = `
Convert cooking instructions from imperial to metric measurements.
Keep the rest of the text unchanged.

Original: "Preheat oven to 350°F. Add 2 cups of flour and 1/2 pound of butter."
Converted: "Preheat oven to 175°C. Add 240g of flour and 225g of butter."

Original: "Bake for 30 minutes at 400°F. Use a 9-inch pan."
Converted: "Bake for 30 minutes at 200°C. Use a 23cm pan."

Original: "Mix 3 tablespoons of sugar with 1 cup of milk."
Converted: "Mix 45ml of sugar with 240ml of milk."

Original: "${instructions}"
Converted:`;

Key Takeaways

1. Few-shot learning teaches by example - show, don't just tell
2. Choose diverse, representative examples that cover different scenarios
3. Maintain consistent formatting across all examples
4. Include edge cases to handle ambiguous inputs
5. Order matters - put important patterns near the end
6. Balance token usage - more examples cost more but may improve accuracy
7. Test and iterate - validate your prompt with real-world inputs

Resources

Next Lesson

You have learned to teach AI by example. In the next lesson, you will learn chain-of-thought prompting - a technique that helps AI solve complex problems by thinking step by step.

Continue to Lesson 3.4: Chain of Thought