Conditional codec
Every absolute single-stream number is entropy-floored — which is why natural data ties a general compressor. Every uncapped win is conditional: H(X | reference) ≪ H(X). The conditional codec makes that the default: register a shared reference once, then code everything relative to it.
How it works
- Register a reference once — it is stored under $AT1_HOME/refs and every conditioned archive names it.
- Compress keeps the smallest byte-exact of three encodings (never-worse): numeric-delta when the input ≈ the reference array (a sensor day vs baseline, a fine-tuned checkpoint vs its base), shared-dictionary when the input shares substrings with the reference (a version, a near-duplicate, a corpus member), or unconditional when the reference does not help.
- Decompress rebuilds from the payload + the reference; it fails cleanly if the reference is missing — never silent corruption.
CLI
at1 conditional register baseline_day baseline.txt --numeric
at1 conditional compress today.txt today.at1c --ref baseline_day --numeric at1 conditional decompress today.at1c today.out # bytes / versioned files (shared-dictionary): at1 conditional register doc_v1 v1.bin at1 conditional compress v2.bin v2.at1c --ref doc_v1 at1 conditional decompress v2.at1c v2.out
What it collapses (byte-exact)
| Item | Conditioned on | Gain | Mode |
|---|---|---|---|
| sensor day | baseline day | ~1.4× | delta |
| fine-tuned weights | base checkpoint | ~4.9× | delta |
| document v2 | v1 | ~48× | shared-dict |
| corpus member | corpus prior | ~66× | shared-dict |
| unrelated data | baseline | 1.0× | unconditional |
The reference is stored once and amortised across every conditioned item — the natural fit for versioned data, model registries, per-tenant baselines and corpora. at1 conditional compress is a metered encode on the same per-TB axis as every other encode surface.