HN Showcase
Back to browse
GitHub Repository
493 starsRust

Witchcraft and Pickbrain – fast multi-vector semantic search in Rust

by jacobgorm·Apr 16, 2026·6 points·0 comments
Visit ProjectView on HN

AI Analysis

●●●BangerWizardryBig BrainDark Horse

Vector search on SQLite alone hits 21ms p.95 — no vector database required.

Strengths
  • •Single SQLite file stores embeddings, index, and documents — zero external dependencies
  • •21ms p.95 latency on MacBook Pro beats original XTR-Warp on server hardware
  • •Pickbrain demo indexes Claude Code/Codex sessions for agent memory retrieval
Weaknesses
  • •33% NDCG@10 accuracy trails behind dedicated vector databases like Pinecone or Weaviate
  • •Requires manual weight downloads from HuggingFace and GGUF quantization step
Category
AI/ML
Target Audience

ML engineers, developers building RAG applications

Similar To

Chroma · LanceDB · Qdrant

Post Description

Witchcraft is from-scratch Rust reimplementation of Stanford's XTR-Warp (SIGIR'25, https://arxiv.org/abs/2501.17788 ) multi-vector semantic search engine. Witchcraft runs out of a single SQLite database, is blazing-fast (21ms p.95 end-to-end search latency on NFCorpus on a MacBook Pro), accurate (33% NDCG@10), and easy to deploy in your own apps. The Witchcraft repo also comes with Pickbrain, a sample app and agent skill that you can use to instantly query across all your Claude Code and Codex CLI sessions, effectively giving your agents global long-term memory. Please see the Github page for more details, and feel free to ask questions here, and I will try to answer them.

Similar Projects

AI/ML●●Solid

PardusDB – SQLite-like vector database in Rust

Using a single-file .pardus format with CREATE/INSERT/SELECT + SIMILARITY queries gives a very familiar developer UX for embedding storage. The combination of graph-based ANN, full transactions, thread-safety, and zero external dependencies is an uncommon and useful engineering combo for local-first AI work; it would win more attention with benchmark comparisons and richer ecosystem integrations (connectors/clients).

Niche GemWizardry
JasonHEIN
205mo ago