Queryable compression, compared honestly.
Compressing data no longer has to make it unreadable. A small group of formats now keep data queryable while it stays packed: znippy, Vortex, OpenZL and AT-1. They are not the same tool. Here is an honest map of what each one is for, and the one combination only AT-1 covers.
| Format | Smaller on structured data* | Query in place | Verify byte-exact | Tamper-evident | Per-subject erase |
|---|---|---|---|---|---|
AT-1 (TinyFiles) Verified, erasable, queryable format structure-aware ratio below general zstd, queryable in place, SHA-256 verified, tamper-evident, per-subject erasable | |||||
znippy Parallel random-access archive (open) all-core speed, per-file random access, queryable via Arrow/DuckDB, blake3-verified, immutable seal; explicitly does not try to beat zstd on ratio, and seals can't erase one subject | by design: no | seal | |||
Vortex Analytics columnar format (Apache-2.0) a fast next-gen columnar format: much faster scans and random access than Parquet, roughly Parquet-sized; built for hot analytics, not integrity or erasure | ~Parquet size | ||||
OpenZL Structured-compression codec (Meta, open) format-aware codec that beats general zstd on structured data; a compression library to embed, not a queryable or verifiable archive | |||||
Parquet + zstd Columnar baseline the ubiquitous baseline: queryable and great tooling, but no integrity proof and no per-record delete |
* “Smaller on structured data” means below a general-purpose compressor (zstd/xz) on structured inputs. znippy deliberately trades ratio for all-core speed; Vortex targets scan speed at roughly Parquet size; OpenZL and AT-1 are structure-aware and go below general zstd. Ratios depend on your data.
What each one is for
znippy
A fast, open, parallel archive: pack a directory at all-core speed and pull any single file without unpacking the rest, with a queryable index and blake3 verification. Great when you want speed and simplicity in an open tool. It does not aim to beat zstd on ratio, and its immutable seals don't erase a single subject.
Vortex
A next-generation analytics columnar format (Apache-2.0, Linux Foundation): far faster scans and random access than Parquet, at roughly Parquet size. Ideal for hot analytical workloads. It isn't built for integrity proof or per-subject erasure, and isn't trying to be smaller than Parquet.
OpenZL
Meta's open structured-compression codec: it models a file's structure with reversible transforms to beat general zstd on structured data, decoded by one universal decompressor. A strong library to embed in your own pipeline. It is a codec, not a queryable or verifiable archive.
AT-1 (TinyFiles)
The one that covers the full set at once: structure-aware compression below general zstd, queryable in place, byte-exact SHA-256 verification, tamper-evidence, and per-subject GDPR/POPIA erasure, with engine integrations and support. If you need stored data to stay small, searchable, provable, and erasable, that combination is AT-1's lane.
Queryable-compression questions, answered
- What is the best queryable compression format?
- It depends on the job, and the honest answer is that a few good tools now exist. znippy is a fast, open, random-access archive you can query with DuckDB or Polars. Vortex is a fast analytics columnar format. OpenZL is Meta's structured-compression codec that beats general zstd on structured data. AT-1 (TinyFiles) is the one that combines four things the others don't do together: it stores structured data smaller than general-purpose zstd, keeps it queryable in place, proves it restores byte-for-byte, and lets you erase one data subject for GDPR/POPIA. Pick by what you need: raw speed, analytics, a drop-in codec, or a verifiable and erasable store.
- AT-1 vs znippy, which should I use?
- They share a philosophy: a verified, queryable, random-access archive. znippy is excellent when you want all-core speed, an open-source Rust tool, and simplicity; it chunks data with zstd and is upfront that it does not try to beat zstd on ratio, and its seals are immutable (no per-subject delete). AT-1 is the choice when you also need the archive to be smaller than general zstd on structured data, and to be per-subject erasable for compliance, with commercial support and published benchmarks across 11 public datasets. Fast open archive: znippy. Smaller-on-structured and erasable: AT-1.
- AT-1 vs Vortex?
- Different jobs. Vortex is a next-generation analytics columnar format built for scan and random-access speed; it is roughly Parquet-sized and has no tamper-evidence, byte-exact verification, or per-subject erasure. AT-1 is a verifiable, erasable storage and archive layer: in cold-tier mode it stores a table smaller than Parquet+zstd, and it adds integrity proof and per-subject erasure, while staying queryable. Because AT-1 reads back as Parquet, the two can even coexist: Vortex for hot analytical speed, AT-1 for a smaller, provable, erasable cold tier.
- AT-1 vs OpenZL?
- OpenZL and AT-1 share the core idea of modeling a file's structure to compress below general-purpose zstd, and OpenZL is a strong, Meta-built open codec for exactly that. The difference is scope: OpenZL is a compression library you embed, with no in-place queryability, verification, or erasure. AT-1 is a full format and platform where structure-aware compression is one layer, alongside query-in-place, SHA-256 verification, tamper-evidence, and per-subject erasure. Want a codec to drop into your own pipeline: OpenZL. Want the stored data to also be queryable, provable, and erasable: AT-1.
- Is AT-1 actually smaller than zstd?
- On structured data, yes, because it is structure-aware rather than a general byte compressor. Across 11 real public datasets AT-1 measured 1.1 to 2.5x smaller than xz-9 and about 23% smaller than Parquet+zstd, each verified byte-for-byte lossless. On already-random or already-compressed bytes there is little structure to exploit and AT-1 falls back to being no worse than the general compressor. The full methodology and the boundaries we don't claim are on the benchmarks page.
Sources
Competitor claims are taken from each project's own documentation, current as of November 2026. AT-1's figures are the verified, reproducible results on our benchmarks page.
Want the one that stays small, searchable, provable, and erasable? Bring a sample and we'll show you on your data.
Get in touch