Properties of a good memory

Apps with no Memory are boring. Compare a static website from the 90s with any SaaS or social network or phone app: the former knows nothing about you, the latter knows a lot. From UI preferences (dark or light mode?) to basic personal data (name?) to your friend list to what things you engage with the most, today's software knows you quite intimately.

In contrast, LLM applications often... don't. ChatGPT has no memory. (OK -- you can now add a few sentences in the UI - but it is still basically static.) Github Copilot keeps some history of how you've interacted with your IDE, but AFAICT none of this carries over from working on one project to another. Perhaps there are deeply personalized applications but I haven't used any yet.

This is an obvious opportunity, but I think there's no good design pattern for it yet. People use basic RAG patterns for memory, and in a chat UI the conversation history serves as extremely short term memory, but I haven't seen an approach that's been demonstrated to work, yet.

I imagine a good memory would combine active and passive modes. In passive mode, throughout interacting with the user, the application would pick up things about you. In active mode, you could explicitly tell it to behave some way in the future.

Writing into the memory should probably be done asynchronously. If deciding what should go into the memory takes time, it doesn't make sense to block the user. The value from long-term memories will come in the future anyway, not in the current interaction.

The memory should probably be normalized. Sort of like how the human brain seems to use Sleep to collate experiences. A hand-wavy analogy is that you start the day with sum(weights) == 1.0, and during the day some weights get increased as experience accumulates. Then sleep serves as the weights := weights / sum(weights) normalization clause.

Maybe there should even be a threshold for long term storage? Like, if a thing has been experienced once, it probably can be forgotten. More than three times, and it has a much better chance of being relevant.

There should definitely be a time decay. Memories farther in the past should get lower weight simply because they're less likely to be relevant.

I think emotional state should factor in? In humans, it definitely does. Some memories burn into us because of elevated levels of (insert hormone). And others slip away because evolutionarily it's better if we don't remember -- giving birth is a well-known example of this.

Or perhaps we shouldn't model this on the human brain at all. We didn't need to imitate birds to make a good flying machine, so maybe we can engineer a good memory from first principles instead of imitation.