GitHub Repository

Build continually improving models on your agent traces by distilling frontier open models

119 starsPython

Distill and serve small models with frontier quality for half the cost

Name: Distill and serve small models with frontier quality for half the cost
Availability: InStock
Author: SilenN

by SilenN·Jul 26, 2026·41 points·21 comments

Visit Project View on HN

AI Analysis

●●SolidBig BrainSolve My Problem

Turns existing OTel traces into a self-improving router against GPT-4.

Strengths

•Leverages existing OpenTelemetry traces instead of requiring new instrumentation.
•Automates distillation and routing policies based on real production data.
•Supports local serving with an OpenAI-compatible endpoint for easy drop-in.

Weaknesses

•Relies heavily on the quality and volume of traces you already collect.
•Complex setup with multiple optimization steps before seeing cost benefits.

Post Description

Hi HN, we built world-model-optimizer, an open-source tool for continually improve models specialized to agents. Today we are launching `wmo serve`, a tool to route repetitive tasks to distilled smaller models.

Agent traces you already capture are opportunities to get signal on how to make your model cheaper, faster, better. We continuously improve - your specialized model through distillation from open source models - model routing to frontier + custom models - token compaction to remove noise and save tokens

Demo: https://www.youtube.com/watch?v=2_m4Ze6mdko

Pass in traces and an OpenRouter key, and wmo starts a local OpenAI-compatible endpoint to run with your model at a lower cost with equivalent quality. Behind the scenes a router decides which tasks should go to the frontier versus your model. Tinker continually trains as new traces arrive.

We also offer a hosted solution for anyone that just wants a frontier quality endpoint with self-improvement over time at a 40%+ lower cost.