Meta-Prompting & Self-Reflection

The best prompt writer might not be you, it might be the model itself. Meta-prompting uses an LLM to write, critique, and improve prompts. The meta-loop: generate a prompt, test it, have the model critique the results, and refine. This lesson covers manual meta-prompting, automated approaches (DSPy), and self-reflection techniques.

What you'll learn

• Meta-prompting: asking an LLM to write or improve a prompt for another LLM call
• The meta-loop: generate → test → critique → refine, same as manual prompt engineering, but the model does the critiquing
• Self-reflection: the model reviews its own output and improves it before returning to the user

Build it

Step 1: Ask the model to write a prompt

You can use the model to generate its own prompts by providing clear requirements and constraints.

meta_prompt = """Write a system prompt for a customer support agent with these requirements:
- Handles refund requests, order tracking, and product questions
- Never issues refunds without order verification
- Responds in 2-3 sentences
- Polite but firm about policies

Return ONLY the system prompt, nothing else."""

generated_prompt = call_llm(meta_prompt)

Step 2: The meta-loop

A meta-loop automates the iterative prompt engineering process by using the model to critique and refine its own output.

def meta_loop(task_description, benchmark, iterations=5):
    prompt = call_llm(f"Write a prompt for this task: {task_description}")

    for i in range(iterations):
        score = evaluate(prompt, benchmark)
        if score >= TARGET:
            break

        failures = get_failures(prompt, benchmark)
        critique = call_llm(f"""This prompt scored {score}% on the benchmark.
It failed on these cases: {failures}

Critique the prompt and suggest specific improvements.
Return the improved prompt.""")
        prompt = critique

    return prompt, score

Step 3: Self-reflection

Self-reflection prompts the model to double-check its initial response and correct potential errors before returning the final output.

prompt = """Answer the user's question. Then review your own answer for accuracy
and completeness. If you find any issues, provide a corrected answer.

User: {question}
Answer:"""

What goes wrong

Mistake	How you notice it	The fix
Model generates overly complex prompts	The generated prompt is verbose and hard to maintain	Add "keep the prompt concise" to the meta-prompt
Meta-loop over-optimizes	Score plateaus, prompt gets longer each iteration	Set a max iterations and a score threshold. Stop when improvements are marginal (< 2%)
Self-reflection agrees with itself	Model says "my answer is correct" even when wrong	Use a different model for the critique step. Claude can critique GPT-4's output more objectively

Next: Prompt Templates & Variables