Connect your tools & pipelines

AT-1 is a queryable file format over your object store, not a database you install. Your .at1 files stay where they are, S3, R2, GCS, a laptop, and the tools you already use talk to them. Everything below is lossless / byte-exact by construction and meters through the same usage you already see.

PostgreSQL wire server, query .at1 from any SQL tool

at1 postgres speaks the PostgreSQL v3 wire protocol, so psql, DBeaver, Metabase, Tableau, or any JDBC/psycopg client connects and queries your .at1 files as SQL tables. No extension to install; the WHERE still pushes down to the file, so a selective query reads only the row-group blocks it needs (and works over a remote URL too). Quote table names that contain - or . (e.g. FROM "fc_trans-260529"). Near-complete analytic SQL: SELECT DISTINCT, COUNT(*)/COUNT(DISTINCT) / SUM/MIN/MAX/AVG, GROUP BY, HAVING, ORDER BY, LIMIT/OFFSET, UNION, INNER/LEFT/RIGHT/FULL/CROSS JOIN … ON, FROM (SELECT …) derived tables, AND/OR/NOT, IN, LIKE, IS NULL, CASE, arithmetic + string functions, and prepared-statement $1params, verified cell-for-cell against SQLite, so a BI tool's generated SQL runs directly. (Read-only; no window functions / CTEs / correlated subqueries / writes.)

at1 postgres --dir ./data          # serve a directory of .at1 files as SQL tables
# then, from ANY Postgres client, psql, DBeaver, Metabase, Tableau, JDBC, psycopg:
psql "host=127.0.0.1 port=5433 dbname=at1 user=at1"
#   SELECT id, amount FROM orders WHERE amount > 100 LIMIT 50;
# FROM <name> maps to <dir>/<name>.at1; the SELECT/WHERE/LIMIT push down to the file
# (only the touched row-group blocks are read). Simple + extended protocol, params supported.

Hosted encoder, encode from any language, no CLI

at1 encode-api is an HTTP encoder: POST the bytes, get a verified .at1back. For anything that can't run the native binary, a browser, a serverless function, another language. The full lossless verification gate runs on every encode, so a non-lossless input is rejected, never silently mangled.

at1 encode-api serve --port 8091           # (add --require-key to meter by API key)
# from a browser, a lambda, any language, no native CLI needed:
curl --data-binary @data.csv \
  "http://localhost:8091/encode?domain=auto&name=data.csv" -o data.at1
# response headers: X-AT1-Original-Bytes / X-AT1-Compressed-Bytes / X-AT1-Ratio / X-AT1-Lossless

Remote-storage watcher, files become queryable on arrival

at1 remote-watch watches an S3 / R2 / GCS prefix and encodes each new object to a queryable <key>.at1 next to it, idempotent, skips anything already encoded. Point it at your landing zone and raw drops become verified, queryable containers with no pipeline step.

at1 remote-watch --bucket my-bucket --prefix raw/    # (R2 creds from env; --endpoint for S3/GCS)
# watches the prefix and encodes each NEW object -> <key>.at1 beside it, idempotent.
# files landing in object storage just become queryable, verified containers.

Byte-exact .zip recompression

at1 zip recompresses the members of a .zip and exploits cross-member redundancy that per-member DEFLATE never saw (16.8× on a set of similar exports in our tests), then reconstructs the original .zip bit-for-bit, it reproduces each DEFLATE stream exactly, with a never-worse raw fallback for anything it can't.

at1 zip pack   archive.zip   archive.at1zip     # recompress members + cross-member redundancy
at1 zip unpack archive.at1zip archive.zip        # reconstructs the ORIGINAL .zip, bit-for-bit
# DEFLATE members are decompressed, recompressed with xz (capturing redundancy per-member
# DEFLATE never saw), then re-DEFLATEd to the exact original bytes on unpack. Never-worse
# raw fallback for anything it can't reproduce (encrypted / zip64 / odd deflate).

CI-friendly metering

Machine-readable usage and a build that doesn't break on a quota wall: at1 usage --json always exits 0, and --soft-fail turns a hard quota deny into a warning so a pipeline keeps going (the event is still recorded).

at1 usage --json                 # {"used_gb":..., "allowed":true, "reason":"ok"}  (always exits 0)
at1 --soft-fail compress qcolumnar big.csv out.at1
#   over quota -> WARNS on stderr and continues (exit 0) instead of failing your build

POPIA / GDPR, tag PII columns at compress time

--subject-cols records which columns are personal data as an additive footer, the file still reconstructs byte-for-byte, and at1 infolists them. The container self-documents its PII shape, so governance and per-subject erasure know exactly what they're touching.

at1 compress qcolumnar customers.csv customers.at1 \
    --subject-cols email,phone,national_id
at1 info customers.at1
#   subject-cols: c1:email, c2:phone, c7:national_id   (PII, tagged at compress)
# stored as an ADDITIVE footer: the file still reconstructs byte-for-byte, and the
# container self-documents its PII shape for POPIA / GDPR governance and erasure targeting.