Long Context & Token Economics

An agent that runs for 30 turns accumulates a conversation history that grows every single call, and unlike a one-shot chatbot response, that history gets resent, in full, on every subsequent call. Left unmanaged, a long-running agent task can quietly turn into one of your most expensive API calls of the day, and eventually it hits the context window limit outright and starts failing. This lesson is about keeping that under control.

What you'll learn

• Why output tokens cost several times more than input tokens, and what that means for how you design agent responses
• Managing a growing conversation history so it doesn't quietly become your biggest cost driver
• When to summarize, when to trim, and when to just let the context grow

The problem: cost that grows with every turn

Every agent loop iteration resends the full conversation so far, the original task, every tool call, every tool result, every intermediate reasoning step. On turn 1, that's small. By turn 20, it can be substantial, and you're paying for all of it again on turn 21. This is separate from (and compounds with) the prompt-caching mechanics covered in Prompting & Context Engineering , caching makes repeated identical context cheaper, but it doesn't stop a conversation from growing in the first place.

Input and output tokens aren't priced the same, either, output tokens typically cost three to five times more than input tokens across providers. That has a real design implication: an agent that's chatty in its reasoning or verbose in its final answers is paying a real premium for it. Constraining an agent to return brief, structured output where that's all you actually need isn't just tidier, it's meaningfully cheaper at any real volume.

Managing a growing history

You have three real options once a conversation history starts getting long, and picking the right one depends on what the agent actually still needs to remember:

Strategy	Good for	Costs you	When to pick it
Let it grow	Short tasks, or tasks where every prior step genuinely matters	Cost grows with every turn; eventually hits the context limit	Tasks that finish in well under the model's context window, no special handling needed
Trim old turns	Long-running tasks where only recent context matters	Risk of losing something from early in the task that turns out to matter later	The task is naturally "recent-focused" , e.g. an ongoing support conversation where turn 1 rarely matters by turn 20
Summarize and replace	Long tasks where the gist of earlier work matters but the exact wording doesn't	An extra model call to produce the summary; a small risk the summary drops a detail	Multi-step tasks (research, multi-file refactors) where "what we found so far" matters more than the verbatim trail of how you found it

Build it: a simple summarize-and-replace pattern

To prevent conversation history from ballooning over long multi-turn interactions, implement a compaction helper function. This function summarizes older intermediate turns while retaining the original user prompt and the most recent messages:

def maybe_compact_history(messages: list, max_messages: int = 20) -> list:
    if len(messages) <= max_messages:
        return messages

    # Keep the original task and the most recent turns; summarize everything in between
    original_task = messages[0]
    recent = messages[-8:]
    middle = messages[1:-8]

    summary = call_model(
        prompt=f"Summarize the key findings and actions from this agent history in 3-4 sentences:\n{middle}"
    )

    return [original_task, {"role": "user", "content": f"[Earlier progress summary]: {summary}"}] + recent

Run this check at the start of each loop iteration, before the next model call, once the history crosses your threshold, compact it down to the task, a summary of the middle, and the most recent turns, so the agent keeps the gist without resending every intermediate step verbatim forever.

What goes wrong

Mistake	How you notice it	The fix
Never trimming or summarizing a long-running agent's history	Cost climbs steadily with task length; eventually a context length exceeded error	Add a compaction step once history crosses a reasonable size threshold
Summarizing too aggressively, too early	Agent loses a detail from early in the task that turns out to matter later	Keep the most recent several turns verbatim; only summarize the older middle section
Verbose agent responses when brief ones would do	Output-token cost is a large share of total spend	Constrain response format explicitly where a short, structured answer is all that's needed
Treating caching and history management as the same problem	Cache stays "hot" but total context still balloons over a long task	They're separate: caching cheapens repeated identical prefix; compaction is what stops the prefix from growing unboundedly in the first place

Confirm it worked

To verify that your compaction algorithm successfully limits context growth and reduces API usage charges, compare the total token consumption of compacted versus uncompacted runs:

# Run a long agent task with and without compaction, and compare total token spend
uncompacted_cost = run_agent_task(task, compact=False)
compacted_cost = run_agent_task(task, compact=True)
assert compacted_cost < uncompacted_cost

Check that the compacted run still completes the task correctly, not just cheaply, a compaction strategy that saves money but drops a detail the agent actually needed is a net loss, not a win.

Next: Building Reusable Skills , turning what an agent learns during a long task into something it can reuse next time, instead of relearning it from scratch.