The Regulated Archive

One archive that is compressed, queryable, tamper-evident, and erasable — all at once.

Regulated data forces a trade-off: compress it and it goes opaque; keep it queryable in a warehouse and you can't erase a subject or prove it was never altered. AT-1's Regulated Archive is a single sealed bundle that does all four — so you query without rehydrating, prove integrity on demand, and honour a right-to-erasure request in milliseconds, without rewriting a byte.

Sealedone artifact · SHA-256 verified
Compressed
3–5× smaller
Queryable
in place, no decompress
Tamper-evident
SHA-256 sealed
Erasable
per subject · GDPR Art.17
One sealed artifact — alter a single byte and it stops verifying.

Compressed

A single bundle, 3–5× smaller than raw — competitive with Parquet on footprint, and it carries the next three properties no columnar format does.

Queryable in place

Predicate/projection pushdown returns exact original rows without a full decompress. A selective window over a clustered column touches under 1% of the file — ~8.5× faster than restore-then-scan at a million rows.

Tamper-evident

A SHA-256 manifest binds the analytic and PII parts. Flip a single byte anywhere and verification fails — the archive is provably the original, or provably not.

Per-subject erasable

Each data subject's PII is encrypted under their own key. A GDPR Art.17 erasure destroys that one key in milliseconds; their pseudonymous analytic rows stay queryable and the archive bytes never move.

Nobody else does all four

We're not the smallest file on this table — Parquet edges us on raw ratio. We're the only one that is also tamper-evident and per-subject erasable, in the same artifact you query.

FormatCompressedQueryable
in place
Tamper-
evident
Per-subject
erasable
Live database
erasable & queryable, but not a compressed retained archive
partial
gzip / zstd / xz
smallest-ish, but opaque — any query or erase means a full restore
Parquet + zstd
queryable & compact, but cannot erase a subject or prove integrity
AT-1 Regulated
all four, in one sealed artifact
Forget the person, keep the data useful

The part that makes this legally and commercially real: erasing a subject removes their identifying PII (name, email, card) — but their pseudonymous rows stay queryable. Your aggregates, fraud models and financial totals stay correct while the individual is genuinely forgotten. In our benchmark, all 1,000,000 analytic rows still queried after the erasure, and the analytic archive's bytes were byte-for-byte identical before and after.

Benchmarked, honestly, on 1,000,000 transactions

4.45×
smaller than raw CSV (one bundle). gzip 3.28×, Parquet 5.10× — and neither can erase or prove integrity.
~8.5×
faster on a selective time-window query (89 ms vs 761 ms restore-then-scan), reading 0.4% of the file.
228 ms
to erase a subject and emit a signed certificate — independent of archive size; the bytes never move.
1 byte
is all it takes to fail verification — tamper anywhere and the manifest catches it.

Honest scope: the query advantage applies to selective queries over clustered columns (it reads only the blocks a predicate touches); a random-column full scan reads everything, same as anyone. Per-subject encryption adds storage overhead, which is why we trail Parquet on raw ratio. That overhead is fixed per data-subject, so the storage win shows up when analytic columns outweigh PII— transaction, event and telemetry data with many rows per person (validated on real public payroll: the analytic part compressed 13.7×, while a thin mostly-names table is the wrong shape and barely beats raw). Build throughput is the current frontier we're widening. Cryptographic erasure (“crypto-shredding”) is an established, regulator-recognised method — our contribution is the unified, queryable, productised archive, not a new cryptographic claim.

One command surface

at1 regulated build txns.json --subject-field user_id --pii email,card_last4 --out arc/
at1 regulated query  arc/ --where amount_cents:240000:250000 --select amount_cents
                                        # queries the compressed bundle in place
at1 regulated verify arc/               # -> integrity: PASS
at1 regulated read   arc/ 1337          # -> subject 1337's PII
at1 regulated erase  arc/ 1337 --signing-key issuer.key --out-cert cert.json
                                        # PII destroyed; analytic rows still query; bytes unchanged
at1 regulated verify arc/               # -> still PASS (manifest re-sealed)

Who this is for

  • DPOs & Legal — close the backup-erasure gap while keeping analytics on retained data.
  • Fintech & payments — query transaction history in place, erase a customer, prove WORM integrity for audit.
  • Healthcare & adtech — keep pseudonymous analytics correct after a subject is forgotten.
  • Long-retention archives — years of snapshots that must stay queryable, provable, and erasable.
A free proof on your own data, in 24–48 hours

Send a representative sample — 100k to 1M rows, with identifiers masked on your side if you prefer. We run build → query → erase → verify and return a one-page report with your numbers: storage vs gzip/Parquet, query-in-place latency and % of the file read, per-subject erasure time with a signed certificate, and an integrity check. No data leaves your control beyond the sample you choose to send, and there's no commitment.

Bring a sample of your regulated data — we'll prove all four on it in a pilot.

Compliance questions, answered

How do you prove personal data was actually deleted for POPIA or GDPR?
On every erasure AT-1 emits a signed proof-of-deletion certificate — subject ID, timestamp, a key-destruction proof, and the archive hash before and after (identical). It is admissible evidence to a regulator that the specific record is permanently irrecoverable, not merely that a deletion was requested.
Can you erase one person's data from a compressed backup or archive without restoring the whole thing?
Yes. Each data subject's PII is encrypted under its own per-subject key; an erasure destroys that one key in milliseconds. Nothing is decompressed or rewritten, the archive bytes never move, and every other record stays intact and queryable.
Is cryptographic erasure (crypto-shredding) accepted for GDPR Article 17 and POPIA section 14?
Crypto-shredding — rendering data permanently irrecoverable by destroying its key — is an established, regulator-recognised erasure method. It is also the only one compatible with a tamper-evident archive: physically excising bytes would break the very integrity proof a regulator also wants. Our contribution is the unified queryable archive, not a new cryptographic claim.
Can you query the archive without decompressing or restoring it first?
Yes — predicate and projection pushdown return exact original rows in place. A selective window over a clustered column reads under 1% of the file, about 8.5× faster than restore-then-scan at a million rows.
How do you prove the archive itself has not been tampered with?
A SHA-256 manifest binds the analytic and PII parts. Change a single byte anywhere and verification fails — so the archive is provably the original, or provably not.

How the archive gets small

Every codec tier below is a setting on one command, not a separate product. `at1` picks the best one for your data automatically and is never worse than zip; these are what it is choosing between, and what to reach for when you want to choose yourself.

Adaptive compression

AT-1 picks the best transform and entropy back-end for every numeric column automatically…

Agent State Sync

Sync evolving agent state — KV-cache, long-term memory, context — across a fleet by sending the delta from the prior state, not the whole thing.

Bounded mode

AT-1 Bounded mode shrinks scientific and sensor data far past lossless by honoring a guaranteed maximum error you set…

Codec Compiler

Point AT-1 at a delimited file and it infers a per-column codec — int delta-varints, fixed-decimals, dictionaries, text…

Codec Registry

A versioned catalog of byte-exact codecs for proprietary formats — WinBranch DBF, MT940 SWIFT, EBCDIC copybooks.

Derived-column eliminator

AT-1 auto-discovers columns that are exact functions of their siblings (Total = A + B, running sums…

Format-aware compression

Format-aware, columnar stream-split compression for structured telemetry, CSV, JSON and logs.

FP-CODEC

FP-CODEC is a verified deterministic context-mixing codec for cold text and log archives.

Docs & commands →

GenDerive

Discover the columns that are exact functions of other columns (total = qty×price, tax, buckets, keys, indexes), store them as a formula instead of data…

Graph & set coder

When data is a set — graph edges, adjacency, posting lists, port/ID allowlists — the arbitrary order of entries costs real bits.

Link

When both endpoints share a model of the traffic, a message need only encode its deviation from what the model expects.

LM-Codec

LM-Codec range-codes a language model's per-token distribution to reach the smallest byte-exact archive of your high-value text and code.

Media

Address any frame, query by scene, extract a clip, and verify integrity — straight from a compressed media file, without decoding the whole thing.

Docs & commands →

Portfolio Codec

Compress a CSV column by column: each column is stored as whichever byte-exact transform comes out smallest (never-worse)…

Prompt Compiler

A drop-in proxy that strips the repeated content out of your AI prompts before they're sent — fewer tokens, same answers, no change to how you work.

Queryable, verified video

Your video lake is opaque and unauditable.

Recover

At1 recover finds the exact integer generator behind a stream — LCG, linear recurrence, multiplicative, LFSR…

Replay

When data came from a deterministic generator — a seeded simulation, a synthetic-ML dataset, a Monte-Carlo run…

Docs & commands →

Rule-Tier

For law-governed signals, AT-1 keeps only the generator plus a holdout-validated residual certificate and regenerates on demand within the certified bound…

Docs & commands →

Erasure, retention and confidentiality

Deleting a record from a compressed archive — and being able to prove the deletion happened — is the capability regulators actually ask about. POPIA s.14, GDPR Art. 17, and every retention schedule that has to survive an audit.

Compliance-grade storage

AT-1 stores regulated records compressed, queryable in place, and integrity-sealed: a hash-chained audit trail where a single altered byte is detected AND located.

Right-to-erasure for compressed archives

Erase one data subject from a compressed backup archive in seconds, without restoring or rewriting it.

Docs & commands →

Vault

Share sensitive data as scoped, revocable, audited query access — counterparties get answers, not the bytes.

Append-only, sealed, and still readable in 2050

One capability wearing several names: a log you can add to but never quietly rewrite, sealed at the moment of writing, and readable without us.

Century

A .at1century file contains its own English self-description, a tiny decoder as source, a SHA-256 of the original, and the compressed payload.

Ledger

Append events without rewriting, query and aggregate them in place without decompressing, and prove that no record was inserted, deleted, reordered, or altered…

Docs & commands →

Seal-at-Source

A single small .exe that watches folders on a back-office PC, hash-chains each nightly file and ships it exactly once.

Snapshot Archive

Delta-chain near-identical nightly CSV snapshots into one archive.

WORM

The archive IS the write path.