EvalLens – Open-source tool to evaluate structured LLM outputs
Schema conformance checks beat generic text evals for JSON-heavy LLM pipelines.
The core API to enable metadata-first AI in your projects.
Sends schema not data to LLM, unlike Vanna, but browser performance millions rows remains dubious.
Privacy-conscious developers, enterprise data teams
Vanna.ai · PandasAI · LangChain
There are two common ways to analyse your data with AI ("Talk to your Data"): 1. Use powerful frontier models on the cloud — but hand over your data and pay per token 2. Use on-premise or local models — but accept weaker results and significant hardware costs
LocalFlow proposes a third approach I call "metadata-first AI": send only the schema to the LLM (column names, types, a few stats — never the actual values), and ask it to generate code that runs locally on your full dataset (scale it to millions of rows at zero additional token cost).
I'm curious what you think :)
Live demo (drop in any CSV): https://apps.localflow.fr/demo/ GitHub: https://github.com/localflow-ai/localflow-core
Schema conformance checks beat generic text evals for JSON-heavy LLM pipelines.
Useful calibration dataset, but it's just logged outputs without analysis tools.
Finally separates JSON validity from actual value hallucination in LLM outputs.
Proxy tokens worthless if leaked, real keys never enter LLM context windows.
JSON-to-map compiler with LLM streaming, but Mapbox/Maplibre already own this space.