☃️ Machines That Talk · Day 12

The Long-Chat Snowball

You've met the cash register (Day 11) and the resend trick (Day 5). Now aim the register at a whole conversation and watch the total roll downhill — then learn the one trick pros use to stop the snowball: summarizing.

↓ roll the snowball
☃️ Lab · The Conversation Cost Machine

Set up a chat, watch it snowball

Because the whole history is resent each turn, cost grows faster and faster. Set your conversation and watch each turn's bar climb — the later turns are the tall, expensive ones.

12
40
160
Total cost of the whole chat
$0.00
Last turn alone:
the final turn re-pays for the entire chat before it

💸 Cost per turn:

normal (resend all)after a summary (smaller!)
🐞 Break-It Challenge

The 40-turn avalanche

Tap to push the chat to 40 long turns with no summarizing. Then flip summarizing on and watch the savings.

The lesson: cost doesn't grow in a straight line — double the turns and you roughly quadruple the total, because every turn resends everything before it. Summarizing old turns flattens the snowball into something affordable.
🏷️ The pro wordBecause history is resent, total chat cost grows roughly with the square of the length (call it quadratic growth). Replacing old turns with a short recap is summarization or context compaction — it resets the snowball. The average cost spread over the useful work is the amortized cost.
🎯 Boss Build · Tame the Snowball

Ship a long chat on a budget

Your users have 30-turn conversations (40-token messages, 160-token replies). Your budget is $0.30 per full chat. Find a setup that comes in under budget. Hint: one checkbox changes everything.

set it up above, then check
🚀 Beyond the Basics

How the pros really do it

Real assistants (including coding agents) run for many turns. Here's how they keep the snowball affordable:

📝

Rolling summary (what you built)

Compress old turns into a recap. ✅ Big, simple savings. ⚠️ Fine details fade.

🪟

Sliding window (keep recent)

Only keep the last few turns + a summary. ✅ Predictable cost. ⚠️ Can forget older facts.

💾

Prompt caching (the pro move)

Reuse the unchanged prefix at a discount. ✅ Cheaper & faster repeats. ⚠️ Needs a stable prefix.

🏷️ Pro names to look upContext compaction, rolling / conversation summary, sliding-window memory, and prompt caching. Some tools also store long-term facts in a memory store and retrieve only what's relevant (that's RAG — Day 16!).

🎉 Day 12 Complete!

You watched conversation cost grow quadratically — and tamed it with summarization. Now you know why long chats and coding agents can get pricey, and the exact trick pros use to keep the bill down.

Day 13: The Agent Token Furnace →
Tomorrow: the scariest bill of all — an AI agent that loops, calls tools, and quietly burns tokens each step. 🔥
AI Adventure · Machines That Talk · Day 12 — made for young inventors 🚀