Back to browse
GitHub Repository

A PostgreSQL 18+ extension for streaming tables with incremental view maintenance, powered by differential dataflow in Rust.

112 starsRust

pg_stream – incremental view maintenance for PostgreSQL in Rust

by grove·Feb 26, 2026·3 points·3 comments

AI Analysis

●●●BangerWizardryBig BrainBold Bet

Differential dataflow in PostgreSQL; one row insertion touches only one row of computation.

Strengths
  • Differential view maintenance is genuinely novel to Postgres ecosystem; DBSP/dataflow framework is sophisticated.
  • Trigger-based CDC needs no logical replication config; hybrid WAL fallback for high-write scenarios.
  • Automatic delta query derivation removes manual incremental logic burden.
Weaknesses
  • Early-stage (unflagged as production); PostgreSQL 18 only, limiting adoption window.
  • Requires Rust extension infrastructure; operational complexity vs external streaming pipelines TBD.
Target Audience

PostgreSQL operators managing large analytics or reporting workloads

Similar To

Materialize · Kafka streams · Flink

Post Description

pg_stream is a PostgreSQL 18 extension that keeps materialized views current incrementally — no external infrastructure, no separate streaming pipeline.

You define a stream table (a SQL query + a freshness bound) and the extension derives a delta query that processes only changed rows on each refresh cycle:

SELECT pgstream.create_stream_table( 'regional_totals', 'SELECT region, SUM(amount) AS total, COUNT(*) AS cnt FROM orders GROUP BY region', '1m', 'DIFFERENTIAL' );

One INSERT into a million-row table → pg_stream touches one row's worth of computation. Query it like any regular table.

How it works: - Trigger-based CDC captures row changes into buffer tables inside the same transaction (no wal_level = logical required) - A background worker walks a topological DAG of stream tables and fires refreshes in dependency order - The DVM engine (differential view maintenance, DBSP framework) rewrites your SQL into a delta query at definition time — JOINs, GROUP BY, CTEs, window functions, LATERAL, subqueries all supported - Stream tables can depend on other stream tables; a change to a base table cascades automatically through the entire chain

Written in Rust using pgrx. 1,300+ tests (unit + integration + E2E). dbt macro package included.

GitHub: https://github.com/grove/pg-stream Docs: https://grove.github.io/pg-stream

Similar Projects