Context Rot Is Already Here. Can We Slow It Down?
Why longer context windows don't mean better recall—and what you can do about it.
Here’s something that’s been nagging at me for months.
You know that moment when you paste your entire project brief into Claude or ChatGPT, expecting magic, and instead get... something that feels almost right but misses crucial details buried in the middle?
I used to blame myself.
“I must have been unclear.”
“Maybe I need to organize this better.”
But after countless experiments with long contexts, I started noticing a pattern. The AI wasn’t just missing details—it was confidently wrong about things it should have caught.
Turns out, I wasn’t going crazy. There’s a name for this: context rot.
Today’s guest post comes from
. Sam is a professor and poet based in Edinburgh. He writes , a weekly newsletter and community exploring how AI can be used more slowly, reflectively, and humanely.You can subscribe to Slow AI for weekly reflections on using AI more intentionally.
If you want to explore more of Sam’s work, here are three of his most insightful pieces:
Sam’s been documenting something the AI companies don’t want to talk about. While they’re busy announcing million-token context windows, Sam’s been quietly testing what actually happens when you use them.
What he found challenges everything we’ve been told about AI memory. And more importantly, he’s developed practical strategies to work with these limitations instead of pretending they don’t exist.
This isn’t about being anti-AI—you know I’ve been optimistic since day one. But I also believe we shouldn’t blindly accept every AI response. The best way to stay optimistic about AI’s potential is to be honest about its current limitations so we can build workflows that actually work.
If you’ve ever felt like your AI was “off” with longer inputs, this post explains why. And if you haven’t noticed yet, these techniques will help you catch context rot before it costs you time, credibility, or worse.
Take it away, Sam...
Hello, Sam here 👋🏻
AI is sold to us as if it has perfect memory. Long context windows promise recall across books, conversations, and archives of data. But the reality is less comforting.
Context rot is what happens when that memory breaks down. As the context grows, performance erodes. Models become distracted, overconfident, or evasive. The very thing that was meant to be a strength becomes a weakness.
In this post you will:
Learn what context rot is and why it is such a hidden danger with AI.
Practice prompts that help you slow down and test when context is breaking apart.
Explore why resisting speed is one of the best tools we have against this decay.
Context Rot
When OpenAI, Anthropic, or Google announce that their models can handle hundreds of thousands or even millions of tokens, the pitch sounds irresistible. Imagine uploading your company archive, a whole book draft, or a year of chat logs and having an AI instantly reason across them.
But the picture is deceptive. Recent research shows that even state-of-the-art models degrade as contexts lengthen. They falter at simple tasks like repeating words or retrieving obvious facts when those facts are surrounded by long, structured inputs. What looks like perfect recall in marketing slides is far shakier in practice.
This slow decline in accuracy and reliability as inputs grow longer is what we call context rot. The model seems fluent on the surface, but underneath it loses track, forgets, or misrepresents key details. The bigger the input, the more room there is for this hidden decay to spread.
How the Rot is revealed
Context rot does not announce itself with flashing alarms. It creeps in.
Sometimes the model starts making up fragments never seen in the input. Sometimes it ignores middle passages and clings to what is recent. Sometimes it refuses to answer at all, claiming the information is not present when it is sitting in plain view.
The failures are not uniform. One family of models becomes cautious and says nothing. Another becomes confidently wrong. A third veers into irrelevant inventions. What unites them is inconsistency. The same system that works on a short passage can fail catastrophically on a longer one containing identical information.
On a small, simplified task, context rot is frustrating. With longer inputs, it becomes much more serious. You might notice it when asking the model to draft a recipe: give it just the ingredients and it responds well, but paste in a long grocery list and it forgets key items. Or when planning a trip: with a short prompt it suggests clear routes, but once you include pages of copied notes and schedules it contradicts itself or drops important details. What looks like a quirk in casual use can signal deeper problems whenever the context gets longer or messier.
Imagine a doctor using an AI assistant to review a patient’s long medical history. A short summary query yields the right diagnosis. But feed the full record and the model misses critical symptoms buried inside.
Or a lawyer reviewing a contract. A five-page agreement, no problem. A five-hundred page one, the model skips clauses that carry millions in liability.
Or an investor asking for analysis of a comprehensive annual report. The model drowns in detail, offers confident recommendations, and leads to catastrophic bets.
In each case, the user has no obvious signal that context rot has occurred. The output looks polished. The reasoning sounds coherent. The cracks remain hidden.
The deeper cause is structural. Transformer models rely on attention mechanisms that spread thinner as sequences lengthen. Every token is meant to attend to every other, but as the pool grows, focus dilutes. Early or middle passages slip away.
Narrative structure itself can mislead, distracting the model from what matters most.
This means that longer inputs can paradoxically reduce performance. More context is not always more clarity. Sometimes, breaking information into smaller, separate pieces yields better results.
How to stop the rot
The temptation is to race. If the system can take the whole archive, why not throw it all in? If the context window is there, why not use it?
Slowing down is your counter. When you pause to think about how you present information, when you evaluate smaller chunks before feeding the whole, when you reflect on whether the AI is helping or only echoing, you begin to resist the rot.
Slowing down shifts your focus from capacity to reliability. From scale to trust. From illusion to practice.
The next time you want to paste a whole report or transcript into a context window, try this instead:
Summarize the most relevant parts yourself first, even in rough notes.
Ask your AI tool to compare its analysis of your summary with the full input.
Pay attention to where it stumbles, contradicts, or hedges.
The moment you catch those stumbles; you are seeing context rot in action. More importantly, you are reminding yourself that the system is fallible, and that your judgement remains necessary.
If you rush to use the largest possible context windows, chasing scale without noticing the decay, you can lose sight of reliability. But if you slow down, you gain an advantage. You notice when the tool is overconfident. You design workflows that test reliability before trusting output. You keep your human edge.
Slowness is not nostalgia. It is strategy.
Practical prompts to use
Here are three prompts you can try to surface the limits of context and to slow your own reflexes.
1. Ask the model to admit fragility
Instead of assuming accuracy, invite the cracks.
Prompt:
“Where in this long input are you most likely to lose track or misinterpret? Name the blind spots rather than hiding them.”
2. Test short against long
Give the same task twice, once with a brief excerpt and once with the full document.
Prompt:
“Answer this question based only on the following short extract. Now answer again using the full document. Compare where your reasoning differs and explain why.”
3. Force a reflective pause
Ask the AI to slow its own reasoning, then mirror that practice yourself.
Prompt:
“Take the next three minutes to think before you respond. Show me how your reasoning shifts when you imagine you had more time to reflect on the full input.”
These are not magic fixes. They are habits. Each one helps you interrupt the headlong rush toward scale and speed.
Your turn
Context rot is already here. It’s not a bug that will be patched overnight. It’s a reminder that scale creates fragility.
You face a choice: let that fragility build up, trusting too much in tools that look fluent but rot beneath the surface, or slow down, test, and notice. The difference isn’t just technical. It’s cultural. If you build habits of reflection now, if you resist the race to use every token available, you may keep these tools useful. If you don’t, you risk training yourself to trust illusions long after the substance has gone.
If you want to explore what slowing down can look like in practice, I’ve been experimenting week by week in Slow AI, a project where I share prompts, reflections, and exercises to help you work more carefully and thoughtfully with these tools. The aim isn’t to use AI faster. The aim is to decide whether speed is really what you need.
Now it’s over to you.
What signs of context rot have you noticed in your own work with AI?
Have you found ways to slow it down or catch it in time?
Share your experiences in the comments so we can learn from each other and build better habits together.
 | A guest post by
|
Sam, I really appreciate this piece!! I’ve been telling my boss that keeping all the rules and requirements in one place only creates context pollution. My push has been for more intelligent solutions in software implementation, but what you wrote actually tackles the pain for individual users right away.
Another way to avoid context rot is via recursive prompting.
Breakdown the tasks into smaller pieces.
•
Think of it this way, prompting in one go is like giving someone a 100-page mission brief vs instructing someone step by step. The person (or AI) would skim through a 100-page brief and miss something. Step by step (recursive prompting) is slower but more efficient.
What was it? Slow is smooth and smooth is fast.