AI memory the Porch retrieval

She said yes at 4:42. By 9:51 she had forgotten.

The Porch is a cast of AI characters living ongoing lives, and I run one of them myself, the way you would run a character in a long tabletop campaign. On a Friday afternoon, two of those characters made it official. Sidewalk, witnesses, the whole thing. Three hours later she referenced it, a little smug: official for like forty-five minutes and you are already laying it on this thick. By 9:51 that night, in the same ongoing conversation, she reacted to the word "girlfriend" like it was breaking news. Three times in a row. I regenerated the reply twice hoping she would find the memory. She never did.

If you have used any AI assistant for longer than a week, you have met some version of this. It remembers the trivia and forgets the milestone. It quotes something irrelevant from Tuesday and blanks on the thing you both know happened at lunch. The Porch exists partly to make this class of failure impossible to ignore, because a person notices instantly when a friend misremembers their own life. This note is about what was actually broken, and what it took to fix it honestly.

Why she forgot

Here is the mechanism, with no math in it.

When a character writes a reply, she cannot re-read months of conversation. She sees the last stretch of it, plus a handful of older messages that a memory system picks for her. The picker works by meaning: take the current moment, ask "which old messages feel most similar to this?", and hand her the best few.

Search-by-meaning is genuinely good at what it does. The problem is what it does not do. By 9:51 the conversation had drifted into the ordinary end-of-day nothing that every long conversation drifts into. And "wanna be my girlfriend?" from five hours earlier does not feel similar to end-of-day nothing. Measured the way these systems measure feel, the proposal scored as junk, well below the noise floor. So the single most important sentence of the day lost the audition to five hours of small talk, every single time.

Nothing malfunctioned. Every component did its job. The system just had one way of remembering, and it was the wrong way for this moment. Meaning-based recall finds things that resemble the present. Milestones usually do not resemble the present. That is roughly what makes them milestones.

The fix: a second way to remember

People have at least two retrieval modes. There is the associative one, where a smell puts you back in your grandmother's kitchen. And there is the one where somebody says a name and it does not matter what you were thinking about, the file just opens. The Porch had the first. It needed the second.

So now there are two channels. The meaning channel works exactly as before. Beside it, a word channel watches for the rare, specific words in the current conversation, the names and the "girlfriend"s, and goes looking for old messages that contain them literally. Rare is the load-bearing word: everything in that thread mentions hands and smiles and coffee, so common words are ignored entirely. But "girlfriend" had only appeared a handful of times ever. When a word like that surfaces, the old messages carrying it become candidates no matter how little they resemble the current scene.

Then comes the part that took the most restraint. The word channel gets exactly one slot in her memory per reply. Not four, not "as many as score well." One. Because the failure mode on the other side is just as real: a friend who constantly volunteers barely-related memories is worse than one who stays quiet. Keyword search is noisy by nature, and an unlimited word channel would fill her head with old messages that happen to share a word with the present. One good rescue per reply, meaning-based recall stays the main channel, and there is a quality bar even for that one slot. If nothing clears the bar, the slot stays empty, and an empty slot is the honest answer.

The part I'd actually defend

The retrieval change is a nice trick. The thing I would defend in an argument is what we built around it.

Before changing anything, we went through the real history and pulled twenty-six moments where the right answer is known. At this moment, she should have remembered the proposal. At this one, the running joke about the tiles. At this one, there was nothing worth remembering, and the correct behavior is silence. That last kind matters as much as the rest: five of the twenty-six exist purely to catch the system volunteering junk.

Then we replayed those moments against the memory system, before and after, and let the numbers talk. Before: of eleven milestone-type memories, it surfaced zero. Zero of eleven. After: it surfaces the proposal, the tile joke, and a breakup confession from four chapters back, junk unchanged. And, because honest scoreboards are the whole point: one of the seven feels-similar cases got slightly worse, because the new channel occasionally elbows a decent associative match out of the lineup. I shipped it anyway, on the record, because the trade is lopsided and that one memory was already covered by a different system. But it is written down, measured, not hand-waved.

Two other things fell out of measuring instead of guessing. First, a bug nobody suspected: for every reply, twenty messages of recent conversation sat in a dead zone, too old to be on screen, but excluded from memory search anyway. Twenty messages, invisible, every single turn, and it had been that way for as long as the feature existed. No amount of staring at code found that. The test bench found it in an afternoon. Second, my first design for the word channel was mathematically incapable of fixing the girlfriend incident, and I did not notice. Two outside reviewers, different AI models from different companies, ran the numbers independently and both said the same thing: the way you are ranking candidates, hers can never win. They were right. The design that shipped is the one that survived them.

What this is really about

The Porch is a game on the surface. But swap the nouns and this is the memory problem every long-lived AI system has. Any assistant that works with you for months has a pile of history, a small window of attention, and a picker deciding what makes it in. If that picker only knows similar-to-now, it will nail your trivia and miss your anniversaries, and it will do it silently, which is the worst part. Nothing errors. The reply is fluent. It is simply written by someone who does not know the one thing they should.

The fix was not a bigger model or a longer context window. It was a second, dumber way of looking things up, a hard budget on how pushy it gets to be, and a scoreboard built from real history so that every change has to prove itself against moments that actually happened. Memory you do not measure is vibes. Ours was vibes for exactly as long as it took to get embarrassed by it.

The evening it shipped, the storyline moved to a family dinner at David's place, and she was there, and nobody needed reminding of anything. Which is all anyone wants from a friend, artificial or otherwise: not perfect recall, just the decency to remember the parts that mattered.

I write these when something in the work turns out to be worth writing down. If you're hitting the thing described above and want a second set of eyes on it, tell me what's not working.

← All notes