Appearance
Chain-of-Thought Prompting
Adding "Let's think step by step" to a prompt can improve accuracy by 10-40% on reasoning tasks. It's the single highest-leverage technique in prompt engineering. This lesson covers zero-shot CoT, few-shot CoT with worked examples, when CoT helps, and when it just wastes tokens.
What you'll learn
- "Let's think step by step" triggers the model to reason before answering, improving accuracy on complex tasks
- Few-shot CoT provides worked examples of reasoning steps, giving the model a pattern to follow
- CoT helps most on math, logic, and multi-step reasoning; it adds latency and cost on simple tasks where it's not needed
The problem
Ask a model "If a train leaves at 3 PM traveling 60 mph and another leaves at 4 PM traveling 80 mph, when do they meet?" Without CoT, the model might guess. With CoT, it walks through the calculation step by step and gets the right answer. The difference is not in the model's knowledge, it's in giving the model permission and structure to think before answering.
Build it
Zero-shot CoT
Zero-shot CoT simply appends a reasoning trigger to the prompt, encouraging the model to think before answering.
python
prompt = """If a train leaves Station A at 3:00 PM traveling at 60 mph toward Station B,
and another train leaves Station B at 4:00 PM traveling at 80 mph toward Station A,
and the stations are 300 miles apart, at what time do they meet?
Let's think step by step."""Few-shot CoT
Few-shot CoT provides concrete examples of the step-by-step reasoning process, leading to more structured and consistent logic.
python
prompt = """Solve the following math problems by showing your work step by step.
Example 1:
Q: John has 5 apples. He buys 3 more. How many does he have?
A: John starts with 5 apples. He buys 3 more. 5 + 3 = 8. Answer: 8 apples.
Example 2:
Q: A pizza is cut into 8 slices. Sarah eats 3 slices. What fraction remains?
A: The pizza has 8 slices total. Sarah eats 3. 8 - 3 = 5 slices remain. 5/8 remains. Answer: 5/8.
Now solve:
Q: {question}
A:"""Auto-CoT: let the model generate its own reasoning chain
You can also instruct the model to explicitly output its reasoning chain before providing the final answer.
python
prompt = """Solve this problem. Before giving your final answer, explain your reasoning
in detail. Then state your answer clearly as "Answer: [your answer]".
Problem: {question}"""When NOT to use CoT
Identify tasks where reasoning is unnecessary to avoid adding latency and increasing API costs.
python
# CoT adds latency and cost. Skip it for:
# - Simple factual questions: "What is the capital of France?"
# - Translation: "Translate 'hello' to Spanish"
# - Formatting: "Put this list in alphabetical order"
# - Single-step lookups: "What's the weather in Tokyo?"What goes wrong
| Mistake | How you notice it | The fix |
|---|---|---|
| CoT on simple tasks | Response is verbose, slow, and costs more tokens | Only use CoT for tasks requiring reasoning: math, logic, multi-step analysis, comparisons |
| CoT produces wrong reasoning but right answer | The reasoning steps contain errors but the final answer is correct | This happens. Use self-consistency (next lesson) to catch it: run multiple times and take the majority |
| CoT reasoning is too long | Model rambles, context fills up, costs spike | Add "Be concise in your reasoning" or "Limit your reasoning to 3 steps" |
| Few-shot examples are too specific | Model copies the example's exact reasoning pattern even when it doesn't fit | Use diverse examples that show different reasoning approaches, not just one pattern |
Confirm it worked
Verify the effectiveness of CoT by measuring the accuracy improvement on complex reasoning tasks.
python
# Compare CoT vs no-CoT accuracy on a reasoning task
question = "If a shirt costs $25 after a 20% discount, what was the original price?"
no_cot = call_llm(f"Answer: {question}")
cot = call_llm(f"Let's think step by step. {question}")
# CoT should produce the correct answer ($31.25) more reliably
# 20% off means the price is 80% of original: $25 / 0.80 = $31.25