A lakehouse cold tier that’s 27% smaller — and still queried in place
Store a Delta, Iceberg, or Parquet table smaller than Parquet+zstd, byte-for-byte lossless, without changing your catalog or your engines. Your warehouse queries it and cannot tell the difference.
Tier it, format untouched
`at1 lakehouse tier` stores every active data file as .parquet.at1. The Delta log and Iceberg metadata never change — they still name *.parquet.
27% smaller at rest
Real NYC-TLC: 15.73 MB → 11.44 MB vs Parquet+zstd-9, byte-for-byte lossless. Never worse than the Parquet you started with.
Queried in place
Spark, Trino and DuckDB read it through the gateway with no materialised copy. Snowflake and Databricks return byte-identical results.
Measured on real data, not a claim
Official NYC-TLC yellow-taxi data, 1,000,000 rows, 19 columns. Same rows through every path.
| CSV | 108.85 MB | — |
| Parquet + zstd−9 | 15.73 MB | baseline |
| AT-1 | 11.44 MB | 27.3% smaller |
Then, in a real warehouse: Snowflake and Databricks each returned byte-identical results for count, sum(fare), avg(distance), a filtered count and count(distinct location) between the original Parquet and the AT-1-materialised Parquet.
Run it
The first at1 lakehouse downloads the compiled engine (bundles Arrow, ~120MB), verifies its SHA-256, and caches it. No Python, no source.
npm i -g @tinyfiles/cli at1 login --key <API_KEY> # tiering writes, so it needs a free account at1 lakehouse status ./my_delta_table # what's here, and the saving at1 lakehouse tier ./my_delta_table # store each data file as .parquet.at1 at1 lakehouse serve ./my_table # gateway: Spark/Trino/DuckDB read .at1 as Parquet at1 lakehouse rehydrate ./my_table # materialise back to .parquet
What to know before you rely on it
- It saves storage, not compute. The engine scans the same Parquet it always did — the saving is on what you store, not on query time.
- Snowflake needs S3, not R2. Cloudflare R2 hits a Snowflake region-signing defect (no single REGION value satisfies both its metadata and scan paths). Use S3 for the Snowflake path; the gateway path (Spark/Trino/DuckDB) is unaffected.
- The win is structure, not magic.A table already at the entropy wall won’t shrink much. It is never worse than your Parquet, and the benchmarks show where it wins and where it ties.
Common questions
- How much smaller, and is it lossless?
- On real NYC-TLC (1M rows, 19 columns) AT-1 stored the data 27.3% smaller than Parquet+zstd-9, byte-for-byte lossless in both directions. The win tracks the amount of typed, low-entropy structure in your data — decimals, small integers, repeated keys — which most warehouse tables have plenty of. Where a table is already at the entropy wall we say so; the compression is never worse than the Parquet you started with.
- Does the query engine see any difference?
- No. We verified this against real managed warehouses: Snowflake (via an external S3 stage) and Databricks (via a Unity Catalog volume) both returned byte-identical results for count, sum, average, filtered count and count-distinct between the original Parquet and the AT-1-materialised Parquet. The engine cannot tell them apart. Spark, Trino and DuckDB read AT-1 directly through the gateway, in place, with no materialised copy at all.
- Two ways it reads — which applies to me?
- Snowflake allowlists storage, so it reads AT-1 as Parquet materialised into recognised object storage (S3). Databricks, Spark, Trino and DuckDB read through the AT-1 gateway, which serves Parquet on the fly from the .at1 files — genuine query-in-place, no second copy, so the storage saving is pure. It does NOT work against Cloudflare R2, which hits a Snowflake region-signing defect; use S3 for the Snowflake path.
- Does it change my table format or catalog?
- No. The Delta _delta_log and Iceberg metadata are never touched — they still name *.parquet. Only the storage of each active data file changes, to *.parquet.at1. That means your catalog, your table definitions and your engines are all unchanged; you are swapping the bytes at rest, not the format.
- What does it cost?
- Reading, querying, rehydrating and serving are free and need no account. Only tiering — writing a table into AT-1 — is metered, on the same per-TB usage meter as the rest of AT-1, because that is the compression work you are paying for. You pay a slice of what we save you.
- How is it installed, given it's compiled?
- `at1 lakehouse` ships in the CLI; the first time you run it, it downloads the compiled engine (which bundles Arrow, ~120MB) from our public release, verifies it against a published SHA-256, and caches it. No Python, no source, and the download happens once. If you would rather pre-place it, set AT1_LAKEHOUSE to the downloaded binary.