CDC SINK · Postgres

Hot in Postgres, cold in .at1, one SQL surface

A logical-replication target that consumes your Postgres change stream (pgoutput) and continuously seals older changes into immutable .at1 cold segments. Recent partitions stay hot in Postgres; the cold half lives in AT-1. One SQL surface (the AT-1 Postgres wire server, or the shipped FDW) queries hot + cold as one table — the client never knows where a row physically lives. UPDATE resolves to the latest version, DELETE is tombstoned, and any point-in-time snapshot falls out for free.

one surface
query hot (Postgres) + cold (.at1) as a single table
exactly-once
durable watermark survives a mid-seal crash + restart
time-travel
point-in-time snapshot at any LSN, for free
tamper-evident
every cold segment re-verifies against its sha256

Point it at a sink dir and let it seal

at1 cdcsink run ./orders_sink --simulate --at-least-once
#   consumed 223 events, accepted 201; sealed cold segments,
#   committed_lsn=395, hot tail 0 rows, schema v2
at1 cdcsink query ./orders_sink --as-of 100      # time-travel to LSN 100
at1 cdcsink verify ./orders_sink                  # every segment re-verifies (tamper-evident)

The consumer is structured for real pgoutput: open a replication slot, decode Begin/Relation/Insert/Update/Delete/Commit, seal on commit boundaries, and send LSN feedback only after a segment is durably sealed — so Postgres never trims WAL ahead of what AT-1 has persisted.

Exactly-once, even on a crash

A durable committed_lsn watermark is advanced atomically with each seal. Crash mid-seal and restart: the half-written segment is discarded, the source replays from the watermark, and the final table state is bit-identical to a clean run. Proven in tests.

Time-travel & schema evolution

Cold storage is append-only, so an UPDATE is a new version and a DELETE is a tombstone — any as-of-LSN snapshot is just “replay winners up to the cut.” Each segment carries a schema stamp, so a column added mid-stream reads cleanly across old and new segments.

Honest scope

Correctness (unified query, exactly-once, crash-recovery, tamper-catch, schema evolution) is proven end-to-end on a stdlib CDC simulation. The cold-half storage win depends on your data — at small scale AT-1 storage is on par with xz. See the note below before promising a number.

On the “~98% cheaper cold half” claim

The mechanic — move cold partitions out of Postgres pages into compressed .at1 — is what saves storage, but the multiple is data-dependent and not yet confirmed on real production partitions. On small or low-redundancy data, AT-1 storage is roughly on par with xz. We validate the ratio on your own cold partitions before quoting a number; the durable, queryable, tamper-evident, time-travelable properties hold regardless of the compression multiple.

“Exactly-once” means idempotent replay across crash/restart via a durable watermark. “Secure” means integrity + reproducibility (sha256-verified segments), not confidentiality.