AI/ML●●Solid
Tiny long-memory benchmark with Harbor running across Islo sandboxes
Compresses long-memory evaluation into three questions testing recall, updates, and abstention.
Niche GemBig Brain
zozo123-IB
202mo ago
Back to browse
GitHub Repository
0 starsPython
Open benchmark for memory layers vs. pasting full chat history
by canu21·Jul 22, 2026·1 point·0 comments
AI Analysis
●●SolidBig BrainShip It
Proves memory layers cut token usage by 80% without losing accuracy.
Strengths
- •Rigorous 'both-correct' gate ensures token reduction claims aren't inflated by accuracy drops.
- •Uses provider billing data for token counts rather than unreliable local estimators.
- •Open harness allows anyone to verify results on their own conversation exports.
Weaknesses
- •Small sample size of 30 probes limits statistical significance for broad claims.
- •Benchmark specifically validates the author's own LeapMemory product.
Category
Target Audience
LLM application developers and researchers
Similar To
LangChain · LlamaIndex · Mem0
Similar Projects
Developer Tools●●Solid
Token-saviour – agent skill from benchmarking 9 token-saving tools
Benchmark-backed router cuts agent token costs by 70% across four task layers.
Big BrainSolve My Problem
vagkaratzas
2414d ago
Developer Tools●●Solid
Oh – verbatim memory and insights of all your past vibecoding sessions
Surfaces token waste by quoting your own words back at the moment they cost you.
Big BrainSolve My Problem
21J3phy
321mo ago
AI/ML●●Solid
Sentinel – browser agent using 3x+ fewer tokens (open benchmark)
Self-benchmark shows Sentinel uses 57x fewer tokens than browser-use on hard tasks.
Big BrainNiche Gem
isoldex
112mo ago
Developer Tools●Mid
Kreuzberg Comparative Benchmarks
The site weaponizes a compact set of benchmarks — throughput, RAM, cold-start, F1 score and install footprint — and even publishes raw JSON on GitHub, which makes it immediately useful for teams comparing ingestion options. Kreuzberg's Rust implementation posts jaw-dropping numbers against common tools; that's interesting, but the page leaves out crucial reproducibility details (datasets, seed runs, environment configs) you'd want before trusting the magnitude of those gaps.
Niche GemWizardry
nhirschfeld
105mo ago
AI/ML●●●Banger
WMB-100K – Open benchmark for AI memory systems at 100K turns
100K-turn benchmark tests situational memory retrieval where others stop at 600.
Big BrainNiche Gem
wontopos
203mo ago