Connect your tools & pipelines

AT-1 is a queryable file format over your object store, not a database you install. Your .at1 files stay where they are — S3, R2, GCS, a laptop — and the tools you already use talk to them. Everything below is lossless / byte-exact by construction and meters through the same usage you already see.

PostgreSQL wire server — query .at1 from any SQL tool

at1 postgres speaks the PostgreSQL v3 wire protocol, so psql, DBeaver, Metabase, Tableau, or any JDBC/psycopg client connects and queries your .at1 files as SQL tables. No extension to install; the WHERE still pushes down to the file, so a selective query reads only the row-group blocks it needs (and works over a remote URL too). Quote table names that contain - or . (e.g. FROM "fc_trans-260529"). Near-complete analytic SQL: SELECT DISTINCT, COUNT(*)/COUNT(DISTINCT) / SUM/MIN/MAX/AVG, GROUP BY, HAVING, ORDER BY, LIMIT/OFFSET, UNION, INNER/LEFT/RIGHT/FULL/CROSS JOIN … ON, FROM (SELECT …) derived tables, AND/OR/NOT, IN, LIKE, IS NULL, CASE, arithmetic + string functions, and prepared-statement $1params — verified cell-for-cell against SQLite, so a BI tool's generated SQL runs directly. (Read-only; no window functions / CTEs / correlated subqueries / writes.)

at1 postgres --dir ./data          # serve a directory of .at1 files as SQL tables
# then, from ANY Postgres client — psql, DBeaver, Metabase, Tableau, JDBC, psycopg:
psql "host=127.0.0.1 port=5433 dbname=at1 user=at1"
#   SELECT id, amount FROM orders WHERE amount > 100 LIMIT 50;
# FROM <name> maps to <dir>/<name>.at1; the SELECT/WHERE/LIMIT push down to the file
# (only the touched row-group blocks are read). Simple + extended protocol, params supported.

Hosted encoder — encode from any language, no CLI

at1 encode-api is an HTTP encoder: POST the bytes, get a verified .at1back. For anything that can't run the native binary — a browser, a serverless function, another language. The full lossless verification gate runs on every encode, so a non-lossless input is rejected, never silently mangled.

at1 encode-api serve --port 8091           # (add --require-key to meter by API key)
# from a browser, a lambda, any language — no native CLI needed:
curl --data-binary @data.csv \
  "http://localhost:8091/encode?domain=auto&name=data.csv" -o data.at1
# response headers: X-AT1-Original-Bytes / X-AT1-Compressed-Bytes / X-AT1-Ratio / X-AT1-Lossless

Remote-storage watcher — files become queryable on arrival

at1 remote-watch watches an S3 / R2 / GCS prefix and encodes each new object to a queryable <key>.at1 next to it — idempotent, skips anything already encoded. Point it at your landing zone and raw drops become verified, queryable containers with no pipeline step.

at1 remote-watch --bucket my-bucket --prefix raw/    # (R2 creds from env; --endpoint for S3/GCS)
# watches the prefix and encodes each NEW object -> <key>.at1 beside it, idempotent.
# files landing in object storage just become queryable, verified containers.

Byte-exact .zip recompression

at1 zip recompresses the members of a .zip and exploits cross-member redundancy that per-member DEFLATE never saw (16.8× on a set of similar exports in our tests), then reconstructs the original .zip bit-for-bit— it reproduces each DEFLATE stream exactly, with a never-worse raw fallback for anything it can't.

at1 zip pack   archive.zip   archive.at1zip     # recompress members + cross-member redundancy
at1 zip unpack archive.at1zip archive.zip        # reconstructs the ORIGINAL .zip, bit-for-bit
# DEFLATE members are decompressed, recompressed with xz (capturing redundancy per-member
# DEFLATE never saw), then re-DEFLATEd to the exact original bytes on unpack. Never-worse
# raw fallback for anything it can't reproduce (encrypted / zip64 / odd deflate).

CI-friendly metering

Machine-readable usage and a build that doesn't break on a quota wall: at1 usage --json always exits 0, and --soft-fail turns a hard quota deny into a warning so a pipeline keeps going (the event is still recorded).

at1 usage --json                 # {"used_gb":..., "allowed":true, "reason":"ok"}  (always exits 0)
at1 --soft-fail compress qcolumnar big.csv out.at1
#   over quota -> WARNS on stderr and continues (exit 0) instead of failing your build

POPIA / GDPR — tag PII columns at compress time

--subject-cols records which columns are personal data as an additive footer — the file still reconstructs byte-for-byte, and at1 infolists them. The container self-documents its PII shape, so governance and per-subject erasure know exactly what they're touching.

at1 compress qcolumnar customers.csv customers.at1 \
    --subject-cols email,phone,national_id
at1 info customers.at1
#   subject-cols: c1:email, c2:phone, c7:national_id   (PII — tagged at compress)
# stored as an ADDITIVE footer: the file still reconstructs byte-for-byte, and the
# container self-documents its PII shape for POPIA / GDPR governance and erasure targeting.