Witchcraft and Pickbrain – fast multi-vector semantic search in Rust

Name: Witchcraft and Pickbrain – fast multi-vector semantic search in Rust
Availability: InStock
Author: jacobgorm

by jacobgorm·Apr 16, 2026·6 points·0 comments

Visit Project View on HN

AI Analysis

●●●BangerWizardryBig BrainDark Horse

Vector search on SQLite alone hits 21ms p.95 — no vector database required.

Strengths

•Single SQLite file stores embeddings, index, and documents — zero external dependencies
•21ms p.95 latency on MacBook Pro beats original XTR-Warp on server hardware
•Pickbrain demo indexes Claude Code/Codex sessions for agent memory retrieval

Weaknesses

•33% NDCG@10 accuracy trails behind dedicated vector databases like Pinecone or Weaviate
•Requires manual weight downloads from HuggingFace and GGUF quantization step

Post Description

Witchcraft is from-scratch Rust reimplementation of Stanford's XTR-Warp (SIGIR'25, https://arxiv.org/abs/2501.17788 ) multi-vector semantic search engine. Witchcraft runs out of a single SQLite database, is blazing-fast (21ms p.95 end-to-end search latency on NFCorpus on a MacBook Pro), accurate (33% NDCG@10), and easy to deploy in your own apps. The Witchcraft repo also comes with Pickbrain, a sample app and agent skill that you can use to instantly query across all your Claude Code and Codex CLI sessions, effectively giving your agents global long-term memory. Please see the Github page for more details, and feel free to ask questions here, and I will try to answer them.

Similar Projects

AI/ML●●Solid

PardusDB – SQLite-like vector database in Rust

Using a single-file .pardus format with CREATE/INSERT/SELECT + SIMILARITY queries gives a very familiar developer UX for embedding storage. The combination of graph-based ANN, full transactions, thread-safety, and zero external dependencies is an uncommon and useful engineering combo for local-first AI work; it would win more attention with benchmark comparisons and richer ecosystem integrations (connectors/clients).

Niche GemWizardry

JasonHEIN

204mo ago

Other●●Solid