Examples & integrations

Query with DuckDB

Run SQL over a compressed .at1 with no unpack step — selective queries read only the touched row-groups.

-- DuckDB: SQL straight over a compressed .at1, no unpack step
LOAD 'at1';
SELECT symbol, count(*), avg(price)
FROM read_at1('trades-2026-01.at1')
WHERE ts BETWEEN 1704067200000 AND 1704153600000   -- only touched row-groups are read
GROUP BY symbol;

→ Query from your engine

Load into pandas / Polars

Pull a whole table — or just the columns and rows you filter for — into a dataframe via Arrow.

# pandas / Polars: load (a slice of) an .at1 into a dataframe
import at1_arrow
df  = at1_arrow.to_pandas("trades.at1")                       # whole table -> pandas
sub = at1_arrow.to_polars("trades.at1", columns=["agg_id","price"],
                          where={"agg_id": (100, 200)})        # pushed-down slice -> Polars
at1_arrow.write_ipc("trades.at1", "trades.arrow")             # Arrow IPC -> any engine

→ Arrow & dataframes

Expose to Postgres

Mount an .at1 as a foreign table; query it (and JOIN it) with ordinary SQL. Same core bridges Trino/Spark/Flink via JDBC.

-- Postgres: expose an .at1 as a foreign table, query with normal SQL
CREATE EXTENSION at1_fdw;
CREATE SERVER at1 FOREIGN DATA WRAPPER at1_fdw;
CREATE FOREIGN TABLE trades (agg_id bigint, price float8, qty float8, ts_ms bigint)
  SERVER at1 OPTIONS (filename '/data/ticks.at1');
SELECT count(*), avg(price) FROM trades WHERE agg_id BETWEEN 100 AND 200;

→ Postgres / federation

Drop-in S3 storage

Point any S3 SDK at the endpoint: objects are compressed on PUT and decompressed on GET. A failed round-trip returns 422 — the plaintext is never stored.

# S3-compatible endpoint: compress on PUT, decompress on GET — transparently
AT1_CLOUD_TOKEN=secret python at1_cloud.py serve ./store --bucket at1 \
    --access-key AK --secret-key SK --port 9100

# any S3 SDK works (boto3, aws cli, DuckDB, Spark). The plaintext is NEVER stored:
aws --endpoint-url http://localhost:9100 s3 cp events.csv s3://at1/data/events.csv
aws --endpoint-url http://localhost:9100 s3 cp s3://at1/data/events.csv ./back.csv  # byte-identical

→ Managed cloud

Auto-tier cold files

A watcher that compresses aging files in place (optionally deleting the original after a verified round-trip). Scan for savings first.

# Auto-tier a folder: anything older than 7 days becomes a verified .at1
at1-watch /data/archive --older-than 7d --include "*.csv,*.log,*.ndjson" --verify-ledger

# See projected savings BEFORE you commit (samples files, extrapolates)
at1-doctor scan /data --report savings.html

→ Operational tools

Ingest from a URL

Stream a remote file straight into a verified .at1 — the plaintext is never written to disk.

# Ingest straight from a URL — plaintext never lands on disk
at1 fetch https://data.example.com/events.csv  events.at1  columnar
at1 integrity events.at1     # SHA-256 trailer: decode == original

→ Ingest from a URL

Wire into AI agents

An MCP server so Claude, Cursor, or VS Code can compress and query .at1 files as a tool.

# AI agents (Claude / Cursor / VS Code) — compress & query .at1 via MCP
# add the server, then your agent can call it directly:
{ "mcpServers": { "at1": { "command": "python", "args": ["-m", "mcp_server.at1_mcp"] } } }

→ MCP server

Reactive collections in React (TanStack DB & Query)

Hydrate a TanStack DB collection — or a plain TanStack Query queryFn — straight from a compressed, queryable .at1, then useLiveQuery reactively. Ship a big dataset as a small file; query it in the browser.

Render millions of rows in React (TanStack Virtual)

Wire AT-1's in-browser query WASM into TanStack Virtual: the virtualizer asks for the visible rows, AT-1 decodes only the owning row-group — millions of rows from one static .at1 file, no backend, a fraction of the file read.

Use it in an image pipeline

Compressing raw/scientific/medical images, or wondering why your JPEGs don't shrink? The Images guide has the worked Python (Pillow/numpy) example and the honest “what wins, what doesn't” table.