Codec compiler — at1 codec

Your bespoke delimited formats — the ones no general compressor understands — finally shrink. at1 codecreads a sample and synthesizes a per-column codec: integers as zig-zag delta varints, fixed-decimals as scaled integers, low-cardinality strings as dictionaries, free text verbatim — with one inference path and zero per-format code. It's byte-exact, and it beats general compression because it types the columns a statistical coder can only see as bytes.

Use it

# synthesize a per-column codec for FILE and encode it (never-worse fallback built in)
at1 codec compress telemetry.csv -o telemetry.at1cc
#   984,421 -> 66,173 bytes (4.20x vs zlib, synthesized-codec)

at1 codec inspect telemetry.at1cc     # read the synthesized codec schema it built
at1 codec verify  telemetry.at1cc     # byte-exact + SHA-256 check

What it gives you

  • One path, any format — the same inference handles comma, pipe, tab or semicolon files with different column types and order, no per-format code.
  • Byte-exact + never-worse — every container carries a SHA-256 of the original and self-checks the codec before trusting it; with no structure to find it stores plain deflate/lzma, never larger than a general compressor.
  • Inspectable schema — the model the compressor built is a deliverable you can read with inspect, not a black box.

Honest scope

On a real telemetry format it reaches about 4.2× vs zlib-9 and 2.4× vs lzma-6, byte-exact — the win comes from typing columns, and it is reported per file, never promised as a universal ratio. If the file isn't uniformly delimited it says so and falls back; it never pretends to have found a structure that isn't there.

See the marketing overview at Codec Compiler, and structure discovery for recovering a generator behind a sequence.

← CLI reference