Blobsearch – Object storage and DuckDB based Elasticsearch alternative

Name: Blobsearch – Object storage and DuckDB based Elasticsearch alternative
Availability: InStock
Author: conqrr

by conqrr·Mar 18, 2026·2 points·0 comments

Visit Project View on HN

AI Analysis

●●SolidSolve My ProblemDark Horse

S3 + Parquet + DuckDB for logs—known pattern, but well-executed and cheap.

Strengths

•3.7x compression with Parquet + Snappy beats typical cloud log storage ratios
•Hive-style partitioning by date/level enables fast DuckDB queries under 50ms
•Full data portability—your S3 bucket, open formats, no vendor lock-in

Weaknesses

•No built-in UI or alerting—you need Grafana, Lambda, or similar for observability
•Author admits this is a known pattern, just less commonly implemented

Post Description

Hey all, sharing my latest alternative to elasticsearch, cloudwatch etc that either require heavy cloud tax or cost more for a managed solution. This is a known pattern but maybe not well known enough to write logs to s3 as durable storage with Parquet and query rapidly with DuckDB. This has become the primary way I ingest logs for all my side projects and don't ever have to worry about losing logs and possibly free storage.

Features

Format Agnostic - Works with any JSON log format via configurable field extraction Fast - 28K+ entries/sec ingestion Efficient - Parquet + Snappy (3.7x compression) Quick Queries - DuckDB queries in <50ms on 56K logs S3-Compatible - AWS S3, MinIO, DigitalOcean Spaces, R2, etc. Partitioned - Hive-style partitioning by date/level (no redundant part suffixes) Auto-Flush - Configurable automatic flushing (default: 90s) Dedupe - Optional deduplication

Happy to answer any questions about internals