Meta says Muse Wonder leads on memory: the test is Meta's own, and it fits in one context window

Meta says its new Muse Wonder models lead on memory. In Meta’s own blog chart and model card, MemoryBench reads Muse Wonder 85.6, Muse Wonder Full 85.5, Claude Opus 5.5 81.5, GPT-6 Astra 81.4, Claude Fable 5.1 79.0. The chart’s footnote calls it an “Internal evaluation measuring how well models remember and use a user’s context across a long interaction history.”

We did not run MemoryBench or the model. This is Meta’s own material: the card and blog from Oct 12, the docs read Oct 13. What follows is an audit of the claim, not a replication.

What the test is

The model card says MemoryBench is “an internal benchmark” of recall in conversations of 64K to 100K tokens, with distractor conversations and later updates, scored by an LLM judge. The methodology section adds that internally generated competitor runs “use common settings where practical but remain best-effort: prompts, tools, and runtime may not be optimally tuned for proprietary models.”

Read that line twice. The competitors’ scores come from Meta’s own runs, on settings Meta describes as best-effort. That does not make the scores wrong. It does mean the 85.6 vs 81.5 gap is measured on Meta’s home field, by Meta’s referee.

The window

The card lists the context window at 1M tokens at inference. The API docs give the exact figure: 1,048,576 tokens.

If the test’s input is 64K to 100K tokens, that is up to about 10 percent of a 1M window. Our arithmetic: 100,000 / 1,048,576. So the whole test fits inside one context window. Nothing about this result requires memory to persist anywhere outside the prompt.

What “Active Memory” means here

The blog defines Active Memory as “carrying important context about you across conversations and actively retrieving the details relevant to the task at hand.” Meta’s launch video says the model can “remember what’s important to you over days, weeks, months, and beyond.”

Those are different horizons. The benchmark tests recall inside one long conversation. The marketing talks about memory across days and months. The blog text and the captions of its two narrated videos do not say where memory is stored, and the word “memory” does not appear on Meta’s API docs home page. A high MemoryBench score tells you the model is good at finding needles in a long haystack. It tells you nothing about where the haystack lives between sessions.

Our reading, untested

If the test fits in one window, then MemoryBench is a long-context retrieval test, not a memory-architecture test. That is a useful thing to measure. It is not the same claim as “this agent remembers you.”

The question worth asking of any agent memory claim: does your agent’s memory live in the prompt or in a store? Where memory lives changes what a memory score tells you. A score on a vendor’s own benchmark, run by the vendor, on the vendor’s settings, deserves exactly this kind of audit before it shapes an architecture decision.

Auditing a vendor’s benchmark claim is an eval. The AI Eval Harness ($69) is the template for running one. Link in bio.

This post started as an Instagram post →