Why chatbots stop learning, and a sleep-based fix

Why does a chatbot not know today’s news?

A chatbot is trained once, on a huge pile of text. After that, its knowledge is fixed. The Google researchers call this a “knowledge cutoff” (the date its learning stopped).

A chatbot can still use what you type in the chat. But that only lasts while the chat is open. Start a new chat and it is gone.

Why not just train it again on new facts?

Because of a problem called catastrophic forgetting. When a model learns a new task, it can get much worse at tasks it already knew.

Think of a full notebook. Squeeze in new notes and you may smudge the old ones.

What is the sleep idea?

The researchers split a model’s life into two phases.

In the sleep phase there are two steps. In memory consolidation, the model copies short-term facts into new, extra storage space, so old knowledge is not overwritten. In “dreaming”, it writes its own practice examples, keeps the most useful ones and trains on them.

The blog post says the words “sleep” and “dreaming” are borrowed from neuroscience. The model is a fixed design, and people set the training rules.

Does it work?

The authors say it does on their own tests. These cover learning new facts, translating languages the model had not seen, and reading very long texts. They say it forgot far less than the usual methods they compared against.

What’s the catch?

• These are the authors’ own tests, run by the people who proposed the method.
• It is a research idea, not a feature you can switch on in a chatbot today.
• We read the blog post and the paper’s abstract. We did not open the full paper.

Try it

This step is our idea, not the researchers’. Ask a chatbot about something that happened last week. Then ask it when its training stopped. Does the second answer explain the first?

Sources

• The sleep framework: A new way for LLMs to retain knowledge in continual learning, Ali Behrouz and Vahab Mirrokni, Google Research, 27 October 2026
• Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories, Behrouz, Hashemi, Javanmard and Mirrokni, arXiv, 2 June 2026 (abstract read; full paper not opened)