GitHub Repository

Fast self-hosted embedding engine for search, RAG, and reindexing workloads on NVIDIA GPUs. Built in Rust + TensorRT for teams that care about scale, cost, and control.

0 starsRust

I embedded 685M public texts in 32 minutes (on 8x A100, Rust, TensorRT)

Name: I embedded 685M public texts in 32 minutes (on 8x A100, Rust, TensorRT)
Availability: InStock
Author: ddayanov

by ddayanov·Jun 4, 2026·3 points·1 comment

Visit Project View on HN

AI Analysis

●●SolidWizardryBig Brain

3.6x faster than Hugging Face TEI on same hardware with zero Python overhead at runtime.

Strengths

•Production-verified numbers: 685M texts embedded with cost breakdown at $0.01 per 1M messages.
•Native TensorRT in Rust eliminates Python GIL and runtime serialization bottlenecks.
•60 models supported out of the box with automatic ONNX export and TensorRT compilation.

Weaknesses

•NVIDIA-only — no AMD, Intel, or Apple Silicon support despite growing alternative GPU markets.
•Embedding infrastructure is crowded with TEI, Fastembed, and SentenceTransformers already established.

Similar Projects

AI/ML●●●Banger

IgniteMS – batch text embeddings at 253K msg/s on 8x A100

Beats Hugging Face TEI by 3x with raw TensorRT and zero Python runtime overhead.

WizardrySolve My Problem

ddayanov

3015d ago

AI/ML●●●Banger

Xybrid – run LLM and speech locally in your app (no back end, Rust)

Single Rust library replaces backend servers for LLM + speech in Unity and mobile apps.

Zero to OneSolve My ProblemWizardry

theGlenn

622mo ago

Open Source●Mid

70+ AI Providers Under Same Rust Interface

This brings the Vercel AI SDK ergonomics into Rust with a type-safe LanguageModelRequest builder, #[tool] macros to expose callable tools, streaming text and structured JSON outputs, and compatibility with Vercel UI stacks. The sheer provider count (70+) and ready-made agent tooling are compelling for Rust shops; quality will hinge on per-provider coverage and runtime compatibility, but the docs, examples, and CI indicate serious follow-through.

Niche GemWizardry

usermonksme

103mo ago

AI/ML●●Solid