Markanywhere – A Streaming Processor of Meanings
Event-stream parsing beats regex for streaming LLM tokens in real time.
High-performance, declarative stream processor for delimited text data.
Declarative schema mapping replaces cryptic awk $1 column indexing with named fields.
DevOps engineers and data pipeline builders
miller · xsv · csvkit
The goal was to build something with just enough features to serve a better UX while staying fast enough to stay out of the way.
ps aux | grab -d whitespace -m user,pid,cpu,mem,_:4,start,time,command:gj --json --skip 1 # Result: # {"user":"root","pid":"1","cpu":"0.0","mem":"0.0","start":"Mar10","time":"0:03","command":"/sbin/init"} # ...
Key points:- Readable mappings: Define schemas like <name> to map columns to fields, with support for <name>:N for aggregating multiple columns into arrays, <name>:g for greedy mapping (into arrays), _ to skip columns and even <name>:Nj to join multiple columns into a single field.
- Performance: Zero-copy tokenization in Rust. It processes ~17.1M fields/sec. On my machine, it shapes a ps aux dump into JSON in about 13ms.
- Schema enforcement: By default, it tells you exactly which line failed and why on stderr, rather than silently producing garbled data.
- Small & static: Single 800KB binary, zero dependencies.
I’m sharing it now to see if this "schema-first" approach to shell piping feels as useful to others as it has been for me.
Repo: https://github.com/anwitars/grab
Crates.io: https://crates.io/crates/grab-cli
Event-stream parsing beats regex for streaming LLM tokens in real time.
Self-describing archive blocks with mandatory CRC32 and no fallback tricks.
This is a compact, dependency-free TestBed<MyModel> harness that forces models to predict next-step bitset inputs with deterministic seeds — clever for reproducible, low-level experimentation. Execution is pragmatic (header-only, quick compile, clear API), but there's no showcased model that actually passes the tests and the scope is deliberately narrow, so it’s more of a useful lab tool than a breakthrough benchmark.
WebTransport game streaming on recycled crypto miners hits 40ms latency.
Self-hosted simulator streaming beats BrowserStack on privacy and recurring costs.
Another terminal file viewer when bat and less already dominate this space.