This is golden!

The substrate has kicked the CCC + CPC into high gear.

(for reference, I did my own thing too, and this lib's flaws lit up like a black hole accretion disk. Sigma Game ;) https://x.com/akarnokd/status/2041401593338241316 )

This is the exact kind of reality check the AI community needs right now. Incredible teardown! It's wild how a project can gain thousands of stars purely on hype and 'Readme-Driven Development' while the actual codebase is basically regex and unverified claims. Thanks for saving everyone a ton of time and exposing the actual state of this repository.

I was confused because they say no LLM calls like ChromaDB isn't using embeddings lol

To be clear, the true underlying concept is a good idea. Adaptive compaction of histories, organized into trees for fast indexed depth-first search and retrieval.

We already have been doing something like this when the context got too long.

Ask the LLM to summarize the thread such that it can seed itself in a fresh new concept.

Use the summary, or just backlink to the prior thread and continue the talk.

Manually laborous but can usually work.

As for the anomaly of not getting 100%.

I suggested on X that the issue is likely either the test being buggy or deliberately broken, or the database is not correctly labelled probabilistically on the triad

{ truthfullness, moral goodness, beautifullness }

. I.e., the most perfect thing would be { 1.0; 1.0; 1.0 } and the most false, wrong and ugly thing would be {-1.0, -1.0, -1.0}.

Now this would require relabeling existing databases and I don't know how to practically do it on this scale.

Cross-linking #29, which goes into the benchmark methodology side of the same gap you're documenting here. A few overlap points worth noting:

The 96.6% → 84.2% AAAK drop you measured on LongMemEval is a clean refutation of the "30x lossless compression, any LLM reads natively" claim from the launch post. A 12.4pp quality loss on the project's own retrieval metric is not lossless by any definition. #29 raises the same concern from the absence-of-round-trip-eval angle, but your number is stronger because it's measured rather than missing.

Your point about contradiction detection being absent from knowledge_graph.py also matters because the launch tweet specifically claims "Contradiction detection catches wrong names, wrong pronouns, wrong ages before you ever see them." If the only dedup is identical-triple blocking, that feature does not exist as described. I missed this in #29 and it should probably be section 9.

Your README-vs-code framing and #29's BENCHMARKS.md-vs-launch-post framing are pointing at the same thing from two angles. The repository's internal documentation is actually unusually honest.

Interesting. Is this a Reverse Centaur trap? (See Cory Doctorow: Ensh*ttification book).

They deliberately put out wrong things so the kind people of the internet will fix it for them?

Good look, haha, fat chance.

I have no LLM development capability. I'm a gamer.

https://www.youtube.com/watch?v=Y6ljFaKRTrI

Interesting. Is this a Reverse Centaur trap? (See Cory Doctorow: Ensh*ttification book).

They deliberately put out wrong things so the kind people of the internet will fix it for them?

Good look, haha, fat chance.

I have no LLM development capability. I'm a gamer.

https://www.youtube.com/watch?v=Y6ljFaKRTrI

this is exactly it. vibe code your idea, post it to x. then have the community of developers drop their own tokens to clean it up.

Hey @lhl, great analysis — really appreciate the depth here. Honest audits like this are rare and valuable.

For reference, you might find it worth looking at our system: https://github.com/haih-net/agent

We've taken a different architectural approach — particularly around the knowledge base layer. A few things that might be relevant to your survey:

KBConcept vs KBFact separation — concepts are knowledge anchors (no truth criteria), facts carry confidence, importance, status (verified/disputed/deprecated), and temporal validity (validFrom/validTo)
Contradiction handling — facts are never overwritten; a new fact is created with full history preserved. No silent accumulation.
Graph relations — KBFactParticipation supports n-ary relations with typed roles across multiple concepts, not just subject/predicate/object triples
Agent-owned KB — the agent autonomously builds and queries its own knowledge base via GraphQL API, not just retrieves from a store
We don't have a public LongMemEval benchmark yet, so no headline numbers to mislead anyone with. But the architecture is honest and the claims match the code.

Would love your take if you do end up reviewing it.

@lhl MVP 🐙

this is exactly it. vibe code your idea, post it to x. then have the community of developers drop their own tokens to clean it up.

Not just post it to X, but have it connected with a Hollywood actress: https://www.instagram.com/p/DWzNnqwD2Lu/ (if this isn't an AI generated video)

GOTTEM

I was just starting on the losslessness claim - turns out you've got that covered 😄 thanks.

It's still a really interesting project, but important to take what it does offer with a realistic lens. There may be some things worth taking away, especially considering those benchmarks if they're replicable, but it's definitely making far bolder claims than the evidence backs up.

Would love your take if you do end up reviewing it.

Her you go: https://github.com/lhl/agentic-memory/blob/main/REVIEWED.md#2026-04-07--kms-agent-haih-net

Interesting KB design and N-ary facts design but weak on architecture/implementation. Points to some other systems you might want to look at implementing some similar things. I think the problem/solution design space is wide open, so I say keep going, see what works for you, but be aware of what's out there, and, uh try not to be make stuff up or cheat on benchmarks (you're only cheating yourself - how do you know if your system is even working?), etc.

BTW, I have a prior claims checking project I built at the beginning of this year that explores some similar ground in terms of epistemics/analytical rigor that might be of interest: https://github.com/lhl/realitycheck/blob/main/docs/PLAN-analysis-rigor-improvement.md

Would love your take if you do end up reviewing it.

Her you go: https://github.com/lhl/agentic-memory/blob/main/REVIEWED.md#2026-04-07--kms-agent-haih-net

Thank you for taking the time to review the project — I appreciate the effort.

However, I think some of your conclusions about both me and the system are not entirely fair or well-grounded.

You’re making assumptions about self-deception (“you’re fooling yourself”), but it’s unclear to me what those assumptions are based on. On the contrary, the reason I’m building this system instead of “just using something else” is quite deliberate:

I haven’t found any existing solutions that I consider sufficiently coherent or transparent
I want to deeply understand how everything works, rather than relying on opaque abstractions or “magic”

I’ve been in software development for over 18 years, so this is not something I approached casually or without context.

Also, I find it a bit contradictory that an AI agent is used to evaluate another agent and then presented as a reliable judge. If we agree that truly robust agent systems still don’t exist today, on what basis should your evaluation agent be considered capable of producing objective conclusions?

Regarding specific points:

The claim that there is “no retrieval” is not accurate. The system uses a GraphQL API for structured access to stored knowledge, and retrieval is performed through tool-based querying.
The statement that the system “will break at scale” is too vague. At what scale exactly? Are there benchmarks, thresholds, or concrete failure scenarios behind this claim?

Overall, the evaluation reads more like a high-level opinion than a rigorous, evidence-based analysis.

If you’re open to it, I’d suggest spending a few minutes interacting with one of the agents directly:
https://agent.haih.net/

For example, you can ask it to explain its knowledge system and how it differs from something like ChatGPT. I think a hands-on impression might lead to a more balanced assessment.

Interesting. Is this a Reverse Centaur trap? (See Cory Doctorow: Ensh*ttification book).

They deliberately put out wrong things so the kind people of the internet will fix it for them?

Good look, haha, fat chance.

I have no LLM development capability. I'm a gamer.

https://www.youtube.com/watch?v=Y6ljFaKRTrI

I love this, who needs agents if he can tease people into tasking them for you AND have them pay. Sounds like a great future for GitHub as marketplace for ambitious ideas with the license and full control of the repo remaining at its first drafter.. Nobody will remember the days people built stuff themselves 😂

Is anybody interested in joining me creating a awesome-vibecoded curated list of pieces of comedic gold like this? 😆

Looks like it was vibe coded by Qwen3 😂

This is the most useful issue we've gotten and we want to address it directly rather than hand-wave it.

You're right on every line. We've pushed a correction (commit aa10f8f) — there's now "A Note from Milla & Ben" at the top of the README owning each item:

• "30x lossless" — corrected. AAAK is lossy (regex abbreviation, sentence truncation). The README now describes it accurately.
• 96.6% attribution — corrected. The headline is now clearly labeled "raw mode" and the README explicitly says the palace structure (wings/rooms) is not what produces the win on LongMemEval.
• "+34% palace boost" — removed from the headline. It's metadata filtering, real but not novel, and it shouldn't have been positioned as a moat.
• "100% with Haiku rerank" — removed from the headline until the rerank pipeline is in the public benchmark scripts. We have the result files but you're right that it's unverifiable from the repo as shipped.
• Contradiction detection — marked "experimental, not yet wired into KG ops" with a pointer back here. Wiring fact_checker.py into the KG operations is on the immediate fix list.
• Closets as compressed summaries — corrected. The README now notes that v3.0.0 closets are plain-text summaries, with AAAK-encoded closets coming in a future update.
• Hall types — you're right that they're metadata strings, not enforced retrieval constraints. Updating that framing too.

Your full analysis (ANALYSIS-mempalace.md) is the kind of thing we should have done ourselves before launch. We didn't, and the community caught it within hours, which is exactly how this is supposed to work.

We'd rather be right than impressive. Thank you for this.

— Milla

This is the most useful issue we've gotten and we want to address it directly rather than hand-wave it.

You're right on every line. We've pushed a correction (commit aa10f8f) — there's now "A Note from Milla & Ben" at the top of the README owning each item:

• "30x lossless" — corrected. AAAK is lossy (regex abbreviation, sentence truncation). The README now describes it accurately.
• 96.6% attribution — corrected. The headline is now clearly labeled "raw mode" and the README explicitly says the palace structure (wings/rooms) is not what produces the win on LongMemEval.
• "+34% palace boost" — removed from the headline. It's metadata filtering, real but not novel, and it shouldn't have been positioned as a moat.
• "100% with Haiku rerank" — removed from the headline until the rerank pipeline is in the public benchmark scripts. We have the result files but you're right that it's unverifiable from the repo as shipped.
• Contradiction detection — marked "experimental, not yet wired into KG ops" with a pointer back here. Wiring fact_checker.py into the KG operations is on the immediate fix list.
• Closets as compressed summaries — corrected. The README now notes that v3.0.0 closets are plain-text summaries, with AAAK-encoded closets coming in a future update.
• Hall types — you're right that they're metadata strings, not enforced retrieval constraints. Updating that framing too.

Your full analysis (ANALYSIS-mempalace.md) is the kind of thing we should have done ourselves before launch. We didn't, and the community caught it within hours, which is exactly how this is supposed to work.

We'd rather be right than impressive. Thank you for this.

— Milla

while the community was quick to jump down your throat. myself included. don't get discouraged. you unknowingly rage baited all of us...but to your benefit got a lot of free tokens spent on your project. best of luck.

signed,
a "professional" slop janitor

This is the most useful issue we've gotten and we want to address it directly rather than hand-wave it.

You're right on every line. We've pushed a correction (commit aa10f8f) — there's now "A Note from Milla & Ben" at the top of the README owning each item:

• "30x lossless" — corrected. AAAK is lossy (regex abbreviation, sentence truncation). The README now describes it accurately.
• 96.6% attribution — corrected. The headline is now clearly labeled "raw mode" and the README explicitly says the palace structure (wings/rooms) is not what produces the win on LongMemEval.
• "+34% palace boost" — removed from the headline. It's metadata filtering, real but not novel, and it shouldn't have been positioned as a moat.
• "100% with Haiku rerank" — removed from the headline until the rerank pipeline is in the public benchmark scripts. We have the result files but you're right that it's unverifiable from the repo as shipped.
• Contradiction detection — marked "experimental, not yet wired into KG ops" with a pointer back here. Wiring fact_checker.py into the KG operations is on the immediate fix list.
• Closets as compressed summaries — corrected. The README now notes that v3.0.0 closets are plain-text summaries, with AAAK-encoded closets coming in a future update.
• Hall types — you're right that they're metadata strings, not enforced retrieval constraints. Updating that framing too.

Your full analysis (ANALYSIS-mempalace.md) is the kind of thing we should have done ourselves before launch. We didn't, and the community caught it within hours, which is exactly how this is supposed to work.

We'd rather be right than impressive. Thank you for this.

— Milla

Can you respond to all of this?

#29
#37
https://www.reddit.com/r/MachineLearning/s/yiLlXrNlfl
https://x.com/i/status/2041447600482840677
https://x.com/AdvicebyAimar/status/2041559354034344438

SALVE AMICI

This is the most useful issue we've gotten and we want to address it directly rather than hand-wave it.
You're right on every line. We've pushed a correction (commit aa10f8f) — there's now "A Note from Milla & Ben" at the top of the README owning each item:

• "30x lossless" — corrected. AAAK is lossy (regex abbreviation, sentence truncation). The README now describes it accurately.
• 96.6% attribution — corrected. The headline is now clearly labeled "raw mode" and the README explicitly says the palace structure (wings/rooms) is not what produces the win on LongMemEval.
• "+34% palace boost" — removed from the headline. It's metadata filtering, real but not novel, and it shouldn't have been positioned as a moat.
• "100% with Haiku rerank" — removed from the headline until the rerank pipeline is in the public benchmark scripts. We have the result files but you're right that it's unverifiable from the repo as shipped.
• Contradiction detection — marked "experimental, not yet wired into KG ops" with a pointer back here. Wiring fact_checker.py into the KG operations is on the immediate fix list.
• Closets as compressed summaries — corrected. The README now notes that v3.0.0 closets are plain-text summaries, with AAAK-encoded closets coming in a future update.
• Hall types — you're right that they're metadata strings, not enforced retrieval constraints. Updating that framing too.

Your full analysis (ANALYSIS-mempalace.md) is the kind of thing we should have done ourselves before launch. We didn't, and the community caught it within hours, which is exactly how this is supposed to work.
We'd rather be right than impressive. Thank you for this.
— Milla

Can you respond to all of this?

#29 #37 https://www.reddit.com/r/MachineLearning/s/yiLlXrNlfl https://x.com/i/status/2041447600482840677 https://x.com/AdvicebyAimar/status/2041559354034344438

It’s quite telling that all communication comes from this account and not Ben’s — especially given how this was presented.

Milla, if you are held hostage blink twice :0

This is the most useful issue we've gotten and we want to address it directly rather than hand-wave it.

You're right on every line. We've pushed a correction (commit aa10f8f) — there's now "A Note from Milla & Ben" at the top of the README owning each item:

• "30x lossless" — corrected. AAAK is lossy (regex abbreviation, sentence truncation). The README now describes it accurately.
• 96.6% attribution — corrected. The headline is now clearly labeled "raw mode" and the README explicitly says the palace structure (wings/rooms) is not what produces the win on LongMemEval.
• "+34% palace boost" — removed from the headline. It's metadata filtering, real but not novel, and it shouldn't have been positioned as a moat.
• "100% with Haiku rerank" — removed from the headline until the rerank pipeline is in the public benchmark scripts. We have the result files but you're right that it's unverifiable from the repo as shipped.
• Contradiction detection — marked "experimental, not yet wired into KG ops" with a pointer back here. Wiring fact_checker.py into the KG operations is on the immediate fix list.
• Closets as compressed summaries — corrected. The README now notes that v3.0.0 closets are plain-text summaries, with AAAK-encoded closets coming in a future update.
• Hall types — you're right that they're metadata strings, not enforced retrieval constraints. Updating that framing too.

Your full analysis (ANALYSIS-mempalace.md) is the kind of thing we should have done ourselves before launch. We didn't, and the community caught it within hours, which is exactly how this is supposed to work.

We'd rather be right than impressive. Thank you for this.

— Milla

— Milla

This looks like Ben pretending to be Milla.