# AT-1 / tinyfiles.io — full integration guide (for humans and AI assistants) This single file contains everything needed to integrate AT-1 into an existing stack: install, core commands, and copy-paste recipes for DuckDB, pandas/Polars, Postgres, Spark/Trino, S3-compatible storage, auto-tiering, URL ingest, AI agents (MCP), language SDKs, and images. It is intentionally self-contained so an AI assistant can answer "how do I integrate AT-1 with X?" from one fetch. AT-1 (product name: TinyFiles) is a structure-aware LOSSLESS compression engine and platform. It stores structured data smaller than general-purpose compressors, keeps it QUERYABLE in place (SQL pushdown without decompressing), and seals every archive with an embedded SHA-256 integrity trailer that detects and locates a single tampered byte. Decoding is always free and needs no account. -------------------------------------------------------------------------------- ## Install -------------------------------------------------------------------------------- npm install -g @tinyfiles/cli # the full CLI as a native binary — NO Python (npm/pnpm/yarn/bun) npx @tinyfiles/cli compress qcolumnar data.csv data.at1 # or run without installing Connect an account (a FREE account is required to ENCODE/compress at all — 100 GB-compressed, no card; reading, querying and verifying are always free and never need an account): at1 login --key # key from the dashboard -> API Keys # or, for CI/containers: export AT1_LICENSE_KEY= export AT1_METER_URL=https://tinyfiles.io/api Licensed engines (AT-1DB/GenQuery, generative, weights, diagnostic, etc.) ship as a compiled, license-gated `at1-engines` wheel (never public PyPI, never source). Enable a feature in the dashboard (/engines), or in CI fetch a checksummed, version-pinned wheel with the same API key as a bearer token: curl "https://tinyfiles.io/api/at1/engines/download?feature=at1db&ack=1" \ -H "Authorization: Bearer $AT1_LICENSE_KEY" # GET -> { version, wheels:[{name,url,sha256}], sha256sums_url } Verify each wheel's sha256 before install; the (version, sha256) pair is the audit trail. Full guide: /docs/engine-install. Free tier: a one-time 100 GB-compressed free trial (a lifetime credit, not a monthly or hard cap — add a card and metered usage simply continues). Past the trial, metered at the value-based TB-under-management rate; decoding an existing file is never blocked. -------------------------------------------------------------------------------- ## Core commands (the 90% you'll use) -------------------------------------------------------------------------------- # Compress losslessly (output is byte-compared to the original before it ships) at1 compress qcolumnar trades.csv trades.at1 # tables / CSV (QUERYABLE) at1 compress qjson events.ndjson events.at1 # line-delimited JSON (QUERYABLE) at1 compress auto anything.bin out.at1 # auto-detect; never worse than xz # Decompress -> byte-for-byte identical to the original at1 decompress trades.at1 trades.csv # Inspect any .at1 at1 info trades.at1 # codec, size, ratio, whether it's queryable at1 integrity trades.at1 # re-check the SHA-256 integrity trailer (provably byte-exact) # Query WITHOUT decompressing (only qcolumnar / qjson files) at1 query trades.at1 --where price:42000:43000 --select aggId,price,ts at1 sql trades.at1 "SELECT aggId, price WHERE ts BETWEEN 1704067200000 AND 1704067210000" # Aggregates / GROUP BY / JOIN / ORDER BY run in `at1 sql` and over the wire (WHERE still pushes down): at1 sql sales.at1 "SELECT city, SUM(amount), COUNT(*) GROUP BY city ORDER BY SUM(amount) DESC LIMIT 10" at1 sql trades.at1 "SELECT COUNT(*), AVG(price) WHERE price >= 42000" # Zero-scan (GenQuery, patent-pending): predicates on GENERATED columns (autoincrement ids, evenly-spaced # timestamps, counters) are answered from the recovered closed form + their EXACT selectivity, reading ZERO # stored bytes -- 28,000x-80,000x vs a scan, byte-identical to DuckDB. Scrambled generators fall back to scan. # Query IN PLACE over an object store (S3/R2/GCS presigned URL, any Range-capable host) — no # download, no proxy: the container footer drives byte-range GETs, so a selective query moves KB. at1 sql "https://bucket.r2.cloudflarestorage.com/2024/orders.at1" "SELECT id, amount WHERE amount > 100" # A glob / comma-list of same-schema shards is ONE logical table (partitioned-by-day/region): at1 query "sales/2024-*.at1" --where amount:100:999 --select id,amount # Decimal + money predicates push down first-class ($ / thousands recognised; file stays byte-exact): at1 query orders.at1 --where price:9.99:49.99 KEY CONCEPT — queryable vs. not: only `qcolumnar` (tables/CSV) and `qjson` (NDJSON) produce files you can `at1 query` / `at1 sql` in place. Everything else (generic files, images, video, blobs) is stored byte-exact and verified, but is NOT query-searchable. "Query while compressed" operates on ROWS and COLUMNS, not on pixels or opaque bytes. If `at1 query` says "not queryable," re-compress with `qcolumnar` or `qjson`. -------------------------------------------------------------------------------- ## Beyond compression — the rest of the platform -------------------------------------------------------------------------------- AT-1 is not only a compressor; the same verified, addressable container powers a set of higher-level verbs. All are lossless/verified and honest about scope. # Adaptive "never-worse" compression: auto-pick the best transform per input, # fall back so the output is never larger than a general compressor (brotli/xz). at1 optimize input.bin out.at1 # Training-data attestation (EU AI Act Article 10). Prove what was -- and what was # NOT -- in a training corpus. Absence is the hard direction and the one that matters. at1 trainset seal ./corpus --synthetic "*/synthetic/*" --label "assistant-v3" at1 trainset prove attestation.json ./subject_request.txt -o absence.json at1 trainset verify attestation.json absence.json ./subject_request.txt # -> ABSENT - this record is provably NOT in the sealed corpus at1 trainset verifier -o verify.py # dependency-free; hand this to an auditor # # Publish attestation.json (32-byte root + declared composition, NO corpus data). # Keep attestation.index.json -- needed to PRODUCE proofs, never to check them. # Measured: 1M records seal in 3.4s; absence proof ~3.3 KB; verify 0.27 ms (O(log n)). # Sealing is metered. VERIFYING IS FREE FOREVER and needs no AT-1 software. # HONEST SCOPE: proves the corpus DECLARED, not what a model ingested (process control, # not maths); and that the manifest is UNALTERED, not that it was TRUE when written. # Conditional coding: code a stream RELATIVE to a shared reference (a base # checkpoint, a prior version, a template, a corpus prior). Byte-exact, # never-worse -- unrelated input falls back to unconditional. at1 conditional register base_v1 v1.bin at1 conditional compress v2.bin v2.at1c --ref base_v1 at1 conditional decompress v2.at1c v2.out # Reference NEGOTIATION -- "what does the receiver already have?" Each reference # is summarised once into a 1 KB sketch (MinHash over content-defined chunks); # the sender picks the best one by estimated containment WITHOUT reading any # reference body, or declines outright when nothing helps. at1 conditional refs # the inventory at1 conditional negotiate new.bin # which reference covers this? at1 conditional compress new.bin new.at1c --auto # negotiate, then encode # # Measured on a real HuggingFace model zoo (byte-exact, pre-registered gates): # same weights, different container 7,595,626 B -> 6,000 B (1,266x) # identical tokenizer across models 530,531 B -> 159 B (3,337x) # cross-model / same-family weights no gain -> correctly declined # HONEST BOUNDARY: this finds DUPLICATION, not SIMILARITY. Two different models # of the same architecture family share nothing. It pays on model registries and # mirrors carrying the same weights in several containers, shared tokenizers and # vocabs, re-uploads, and versioned data -- not on "related" data in general. # Regulated Archive — ONE bundle that is compressed + queryable + tamper-evident # + per-subject erasable. Built for regulated data where analytics >> PII. at1 regulated pack ./dataset archive.at1 # then query/verify/erase in place # Sign the integrity manifest at build time. Without --signing-key, `verify` proves the # data matches the manifest but NOT that the manifest is the original one — someone who # edits the data can rewrite the hashes to match. Signing binds them to a key you hold. at1 regulated keygen --out issuer.key at1 regulated build records.json --subject-field patient_id --pii "name,email,mrn" \ --out arc --signing-key issuer.key at1 regulated verify arc # -> integrity: PASS (signed manifest) # FAIL names WHICH part changed, and distinguishes an # unsigned manifest from one whose signature is invalid. # LAYOUT MATTERS MORE THAN THE CODEC HERE. Identifying fields belong in a small subject # table, measurements in a large readings table keyed by subject id. Identity repeated on # every row makes the encrypted-PII envelope dominate: on real-shaped biometric telemetry # (200 subjects x 96 readings/day) the flat layout measured 0.67x vs xz -9e, the normalised # one 1.52x, and the readings archive alone 2.99x — same data, same guarantees, 2.3x apart. # Erasing the subject row destroys the only link to the identity; the readings survive # intact and queryable, keyed by an id that no longer resolves to a person. # Queryable Verified Media Container — find-the-moment, extract the exact frame, # verify + pinpoint a tampered region. Works on real camera video. at1 media find clip.at1 --scene-cut # locate; then extract/verify a frame # Provenance & integrity family (structure-aware, cryptographic): at1 origin file.at1 # attest where an artifact came from at1 registry ... # a signed registry of artifacts at1 recover file.at1 # recover readable data from a partially-damaged .at1 at1 discover series.txt # recover the GENERATOR behind a sequence (recurrence/CA/index-fn), byte-exact; # emits a tiny .at1gen container; random data -> honest "no generator" at1 determinism run.at1 # how reproducible/deterministic a recorded stream is at1 ncd a.at1 b.at1 # normalized compression distance (similarity, no model) at1 replay run.at1 # deterministic replay of a recorded stream/log # Cryptographic erasure (right-to-be-forgotten, even inside an immutable archive): at1 erase archive.at1 --subject # per-subject crypto-shred + Ed25519 certificate; # a RECORDED operation that never blocks decoding of the rest. The certificate carries the # archive hash before/after (identical) + a key-destruction proof, and can BIND an external # commitment (--hash-field: an upstream H(payload)) and a caller issuer (--issuer) — so it # seals back into another ledger/attestation as proof the deletion obligation was honoured. # Verify independently with the public Ed25519+SHA-256 reference verifier (no engine internals). # Predictive-maintenance screen (label-free): the compressor's residual + ratio # is a free anomaly signal, validated on real machine-vibration/tool data. It is a # data-quality / early-warning screen, NOT a medical or safety diagnosis. at1 condition-monitor signal.at1 # Pre-sale compressibility audit — honest STRONG / CAPABILITY / PASS on a dataset # before you commit (samples, extrapolates, no cherry-picking): at1 audit /data # Vector storage/search over compressed data, and a generative knowledge container: at1 vector ... # exact vector store/search at1 living-ai build ./kb # build a verified generative KB container; then `ask` # Connect your EXISTING tools — AT-1 as a queryable file format over any object store, not a DB to install: at1 postgres --dir ./data # serve a dir of .at1 as PostgreSQL tables; point psql / DBeaver / # Metabase / Tableau / any JDBC or psycopg client at it. WHERE pushes down to the file; a near-complete # analytic SQL runs natively — SELECT [DISTINCT], COUNT(*)/COUNT(DISTINCT)/SUM|MIN|MAX|AVG, GROUP BY, # HAVING, ORDER BY, LIMIT/OFFSET, UNION, INNER/LEFT/RIGHT/FULL/CROSS JOIN ... ON, AND/OR/NOT, IN, LIKE, # IS NULL, BETWEEN, CASE, arithmetic + string funcs, and $1 prepared-statement params (verified vs # sqlite3), plus FROM (SELECT ...) derived tables incl UNION-in-FROM. Read-only; no window functions # (OVER) / CTEs (WITH) / correlated subqueries / writes. at1 encode-api serve # hosted HTTP encoder: POST /encode -> a verified .at1 (for clients # that can't run the native CLI — a browser, a serverless function, any language). at1 remote-watch --bucket B --prefix raw/ # watch an S3/R2/GCS prefix; auto-encode each NEW object to # .at1 beside it (idempotent), so files landing in storage become queryable containers. at1 zip pack archive.zip archive.at1zip # byte-exact .zip recompression — recompress members + exploit # cross-member redundancy DEFLATE never saw; reconstructs the original .zip bit-for-bit (raw fallback). # AT-1 Serve-DB (/serve-db): a verified single-binary backend (PocketBase-shaped) where every answer is # proof-carrying and every change is a tamper-evident audit event. Self-host ONE binary; no external DB. at1 serve-db serve ./store # start the backend behind one fail-closed auth gate (bearer or HS256-JWT) # Nine non-gated capabilities behind that single auth gate: # 1. VERIFIED BACKEND — a proof-carrying REST /query, byte-exact time-travel (?asof=), # and a live SSE audit stream: every read carries a proof, every write becomes an audit event. # 2. COMPUTE-ON-COMPRESSED — scan-free derived columns, features, and anomaly scores computed straight # from the compressed store (no decompress-first tax). # 3. OFFLINE SELF-VERIFYING EXPORT — a query becomes an ~11 KB HTML file that re-verifies itself in the # browser with no server; if a byte is tampered, it turns red. # 4. DELETE-A-SUBJECT-WITH-A-CERTIFICATE — GDPR crypto-erasure plus an Ed25519 certificate; every # surviving record stays byte-exact. # 5. SEALED-MODEL CO-SERVING — data + a non-extractable model in one process; the license fails closed. # 6. OEM / WHITE-LABEL EMBED — a partner overlay + a per-tenant verified rollup, under your own brand. # 7. VERIFIED READ-REPLICATION — a follower proves byte-exact faithfulness to the leader from a 32-byte root. # 8. SIGNED TRANSPARENCY LOG — an RFC-6962 CT log for the DB change log, with inclusion + consistency proofs. # 9. CROSS-ORG PSI FEDERATION — two orgs compute a joint sum / intersection while neither dataset is exposed. # Auth fails closed by default. For: teams that want a self-hosted backend where every answer is provable # and every change is auditable — not a database to trust, a database that proves itself. # Hosted verified-services APIs (the web consoles at tinyfiles.io/dashboard call these; each runs the # licensed engine server-side and returns JSON — no engine source is shipped). Verification is FREE and # account-less by design; issuing/reporting ops meter. The Python gateways are query_service/at1_*_server.py. # POST /api/at1/query { file, where, select, limit } -> AT-1DB Cloud: predicate+projection query # over a queryable .at1 (reads only touched blocks) # POST /api/at1/prove { op:"verify"|"range"|"infer"|"group", proof, sealed?, answer?, root?, # sortedness_cert? } -> Proofs-as-a-Service: check exactness / O(logN) # SUM-COUNT-AVG / verifiable SQL GROUP BY # (O(log n) per-group proof over a sorted # commitment, sound vs a malicious sealer via a # sortedness certificate) / proof-of-inference (FREE) # POST /api/at1/regulated { op:"worm"|"disclose", ledger?, export? } -> Regulated Archive: verify a WORM # journal is append-only+untampered, or a redacted # disclosure still matches the original seal (FREE) # POST /api/at1/intel { op:"intel"|"dataset", file?, name?, ref? } -> Compression Intelligence # (tell_me_about, METERED) / Dataset Registry # content-address verify (FREE) # CI / governance: at1 usage --json # this account's usage/quota, machine-readable (always exits 0) at1 --soft-fail compress ... # a quota wall WARNS and continues (exit 0) instead of breaking the build at1 compress qcolumnar data.csv out.at1 --subject-cols email,phone,national_id # tag PII columns (POPIA/ # GDPR): stored as an additive footer (`at1 info` lists them); the file still reconstructs byte-exact. -------------------------------------------------------------------------------- ## Compressed-intelligence products (shipped as `at1 `) -------------------------------------------------------------------------------- Eleven products, each one command in the same CLI, all verified-lossless and pay-as-you-go with a monthly free tier. What each is, who it's for, and what they do: # Make AI cost less at1 ctx # AI Memory (/memory): output-lossless compression of an LLM's context + KV-cache. # Same answers, far fewer tokens. Billed on tokens saved (first 1M/mo free). # `at1 ctx proxy` runs it as a drop-in Analyst/LLM proxy that compresses context # transparently on the wire. For: teams building chatbots/agents with large token bills. at1 features # Features (/compute-on-compressed): pull ML features straight from the compressed # columns — skip decompressing first. Billed per extraction (first 10k/mo free). # For: ML/analytics pipelines paying the decompression tax. at1 link # Model-coupled Links (/model-coupled-links): sender+receiver share a model, so the # channel carries only the residual "surprise". Billed per 1k frames (first 10M/mo free). # For: bandwidth-constrained telemetry, video, edge/IoT. # Understand your data & systems at1 observe # Observability (/observability): anomalies by bits-per-event — a spike in # incompressibility is the incident, no rules to write. Billed per events scanned # (first 5M/mo free). For: SRE, ops, security. at1 appraise # Data Appraisal (/data-appraisal): an MDL valuation of a dataset — how much real # information it holds. Billed per appraisal (first 5k/mo free). # For: data marketplaces, data-asset pricing. at1 explain # Explain (/explain): the Kolmogorov machine — returns the shortest human-readable # program that reproduces the data, or honestly says there isn't one. Billed per # explanation (first 5k/mo free). For: research, discovery, R&D. # Keep data — provably, for a long time at1 century # Century Archive (/century-archive): a container that carries its own decoder — # format-rot insurance. Billed per GB packed (first 10 GB/mo free). # For: long-term archives, compliance, records retention. at1 artifact # Model Artifact (/model-artifact): turns private data into a model + a certified synthetic # twin — marginals and central mass preserved, cross-column correlations directionally # faithful — that is nobody's real data. `at1 artifact twin -o ` emits the twin # CSV; `at1 artifact certify ` proves its fidelity; `at1 artifact leakcheck # ` runs the no-leakage battery on any candidate (a copy of the source # FAILs). Billed per build (first 500/mo free). For: teams under POPIA/GDPR who must test # on private data. at1 ruletier # Rule-Tier (/rule-tier): store the generating rule, discard the rows, regenerate # exactly on demand with a certificate. Billed per certified signal (first 1k/mo free). # For: law-governed / simulation / machine-generated data. # Build & query on compressed data at1 codec # Codec Compiler (/codec-compiler): synthesize a byte-exact compressor for an unseen # proprietary format from a few samples. Billed per compile (first 1k/mo free). # For: engineers stuck with legacy/odd binary formats. at1 lens # LENS (/lens) + App Store (/store): apps that are a single verifiable file — data + # interface + proofs in one, runs offline, every number certified, tamper-refuses. # Flagship AT-1 Sheets (a spreadsheet in one file). `at1 lens daily-pack` renders a # certified daily summary sized for WhatsApp/mobile delivery. Billed per app build (Studio: # 25/mo free). Open apps in-browser at /store; creators keep 85% on paid marketplace sales. # For: analysts and app builders who want a shareable, offline, self-proving tool. -------------------------------------------------------------------------------- ## Regulated-archive & reconciliation products (`at1 `) -------------------------------------------------------------------------------- Eight products for legacy-money / regulated operators (built with a design partner who reconciles retail money for a living). All verified, pay-as-you-go with a monthly free tier; verify/read is free. at1 qsign # Signed Query Receipts (/qsign): sign a query RESULT so anyone verifies the reported number # is real WITHOUT the underlying data — a portable receipt a third party re-derives from the # receipt alone. Signing is metered; verifying is free. For "prove this figure" reporting. at1 fpcodec # FP-CODEC (/fpcodec): verified deterministic context-mixing codec for cold text & logs — # maximum shrink (below zstd and xz), bit-identical across architectures, byte-for-byte # reconstruction anywhere, verified round-trip on every encode. at1 snapshot # Snapshot Archive (/snapshot-archive): delta-chain a folder of near-identical nightly CSV # snapshots into one archive. `at1 snapshot asof ARCHIVE 2026-05-14` materializes that # night ROW-EXACT for querying, or `--exact` reproduces the ORIGINAL FILE byte-for-byte # (sha256 matches); no restore; per-night hash-chain tamper-evidence. ~2-3x over gzipping # each night, growing to ~12x on hundreds of high-overlap nights (a solid xz is smaller on # bytes but can't do AS-OF). CSV table snapshots (export HSTs/SQL dumps to CSV first). # `at1 snapshot build-raw` delta-chains BINARY (non-CSV) nightly dumps as a byte-exact # CDC chain. Billed per snapshot packed (first 1k/mo free). For: anyone with a nightly-backup folder. at1 reconcile # Reconciliation Certificates (/reconciliation): `at1 reconcile run A.csv B.csv --rules R` # (inputs are CSVs with date/amount columns) emits a signed verdict (matches M + residuals # X) an auditor re-verifies WITHOUT raw rows; a silently-dropped unmatched row is caught # (partition completeness). Matches on AMOUNT+DATE by default (bank & POS tapes share no # reference strings) — shared tokens optional booster; N:M settlement; k-opening (not ZK). # `at1 reconcile run ... --fees` is fee-aware and composition-tolerant (settles net-of-fee # tapes where the gross/fee split differs). `at1 reconcile grossnet txns.csv credits.csv` # is the DETERMINISTIC gross==net reconciler: dedup (configurable key tuple) -> group -> # optional adjacency merge -> match each batch's gross sum to its bank credit EXACTLY (to # the cent, since real books settle net==gross), with an HONEST evidence split — per-row # observable vs counting-evidence vs on-account, never conflated. Billed per certificate # (first 100/mo free). For: bank/POS settlement forensics. at1 databom # Derivation Certificates / DATABOM (/databom): `at1 databom seal prog.py --in a.csv # --out out.bin` mints a signed, RE-EXECUTABLE cert {input hashes, program hash, output # hash, env fingerprint}. `at1 databom verify cert.json` re-runs the sealed program on the # sealed inputs and confirms the output reproduces byte-exact (tamper of input/program/ # output -> REJECTED, no silent pass). `at1 databom graph c1 c2 ...` composes certs into a # hash-continuous pipeline DAG (one stage's output hash == the next stage's input hash), # verified transitively; a swapped intermediate breaks the chain. `at1 databom refuse-check` # runs a step twice and REFUSES a non-deterministic one (fail-closed; DATABOM's own check, # not the patent-gated replayckpt). Custody proves what arrived, reconciliation proves a # join verdict — DATABOM proves DERIVATION. Signed with ed25519 (TOFU), filed in the codec # registry catalog. Billed per sealed cert (first 100/mo free); verify/graph/refuse free. # For: AI-Act / regulated data lineage — "prove how this figure was produced." at1 disclose # Selective Disclosure (/selective-disclosure): `at1 disclose export SEALED --mask PAN` # emits a redacted view that still verifies against the ORIGINAL seal — prove a value was # present-but-masked, not fabricated; salts in an erasable vault so masked values are also # crypto-erasable. `at1 disclose profile` applies cross-file consistent tokenization (the # same real value maps to the same token across exports). Billed per export (first 100/mo # free). For: PCI/POPIA exports. at1 worm # WORM Journal (/worm-journal): the archive as the live WRITE PATH — per-record hash-chained # streaming append + periodic fsync'd seals + torn-tail crash recovery + `at1 worm anchor` # (publish the seal head; a full re-forge is caught). Rolls sealed segments into the Ledger. # Billed per 1k records (first 100k/mo free). For: POS/pharma/gaming compliance. at1 complete # Completeness / Absence proofs (/completeness): `at1 complete prove-complete ARCHIVE --key # receipt_no --range 1..N` proves every key is present or lists the EXACT gaps; `prove-absent` # proves a record does not exist — sorted-Merkle over a keyed column, succinct (no full # keyset disclosure). Keyed = feasible; arbitrary absence = a soundness wall. Billed per # proof (first 100/mo free). For: deleted-transaction fraud. at1 cdcsink # Postgres CDC Sink (/postgres-cdc): continuously seal a live Postgres into queryable .at1 # cold segments — hot in PG, cold in .at1, one SQL surface, exactly-once across a mid-seal # crash (durable committed_lsn). A Supabase mode seals a hosted Supabase Postgres the # same way. Billed per 1k seals (first 100k/mo free). at1 codereg # Codec Registry (/codec-registry): a versioned catalog of byte-exact codecs for proprietary # formats (MT940/DBF/EBCDIC founding entries, plus a FoxPro/dBase .dbf codec), each CERTIFIED # by byte-exact round-trip on a held-out corpus — makes Century Archive's format-rot insurance # credible. Billed per certification (first 100/mo free). at1 fingerprint # Fraud Fingerprints (/fraud-fingerprints): per-entity compression baselines (per-cashier/ # per-pump) -> a CALIBRATED anomaly REVIEW QUEUE. Two nulls every run + a controlled # false-alarm rate; refuses to emit alerts if the signal fails its own null. NEVER a fraud # verdict — every alert needs a human. `at1 fingerprint pilot` runs a scoped, time-boxed # pilot that calibrates the queue on a site's own history before go-live. Billed per 1k # entity-scans (first 50k/mo free). -------------------------------------------------------------------------------- ## Back-office capture, custody & extraction products (`at1 `) -------------------------------------------------------------------------------- Seven newer products (2026-07) that push the verified pipeline all the way to the point of data capture — the fragile back-office PC, the ingestion boundary, the fuel forecourt, the NL question. All verified, pay-as-you-go with a monthly free tier; verify/read is free. at1 sealagent # Seal-at-Source Agent: a capture agent that watches folders on a fragile back-office PC # and hash-chains + ships each nightly file EXACTLY ONCE — offline / power-cut tolerant, # with a heartbeat. `at1 sealagent run|once|verify|status|init-config`. Seals data where # it's born, before the network can drop or duplicate it. For: distributed retail/branch # sites whose PCs lose power and connectivity. at1 custody # Ingestion Custody Receipts: every ingested file becomes ONE hash-chained append in a # per-site ledger (source, content SHA, row count, timestamp); `at1 custody verify` detects # AND localizes a tampered entry, and it renders an embeddable "Data custody: verified" # badge. Chain-of-custody at the point of intake. For: anyone who must prove what arrived, # from where, unchanged. at1 crossview # Cross-file Views: a manifest-aware UNION of an archive of per-unit .at1 files as one # logical table (e.g. cashup_*.at1 -> view `cashup`), with per-file min/max PRUNING so a # predicate only touches the files that can match. `at1 crossview build-manifest|query| # explain|views`. Query a fleet of per-site/per-day files as a single table. For: operators # with thousands of small per-unit exports. at1 askreceipt # Answers-with-Receipts: NL->SQL where EVERY answer carries a signed, re-verifiable receipt # of the exact SQL run (tables, columns, row-count, result-hash) — a wrong-column answer # becomes a visibly-wrong, auditable query instead of a confident hallucination. For: anyone # letting non-analysts ask questions of regulated data. at1 docextract # Certified Document Extraction: learn a template from sample reports/statements, extract # cells BOUND to their source page + byte offset with a certificate, and TRIP a drift alarm # when a document stops matching its template. `at1 docextract learn|extract|verify`. The # extracted numbers stay provably tied to the source bytes. For: teams keying figures off # PDFs/statements who need to prove they didn't mistype. at1 wetstock # Wet-Stock Reconciliation: statistical fuel-tank inventory reconciliation — detect leaks and # meter drift from daily dips / deliveries / pump totals via a model-coupled residual + # CUSUM; emits a CERTIFIED report with loss rate + confidence interval and an onset date. # For: fuel retailers / forecourt operators (environmental + shrinkage). at1 embed # OEM / Embed Program: run AT-1 engines PER TENANT, roll up ONE consolidated invoice, render # white-label "Verified by AT-1" verify pages, and generate a wholesale price sheet. `at1 # embed run|rollup|verify-page|price-sheet|serve`. For: MSPs / platforms embedding AT-1 # under their own brand. -------------------------------------------------------------------------------- ## Verify any institutional document — selective disclosure with honest trust tiers (https://tinyfiles.io/prove-income · verify at https://tinyfiles.io/verify) Prove a document an institution emailed you (bank statement, invoice, levy/body-corporate statement, municipal bill, payslip, insurance schedule) is genuine, and reveal ONLY the fields you choose — the recipient verifies it in their own browser, nothing uploaded, no account, no AT-1 software. Sender-, document- and country-agnostic BY CONSTRUCTION: the verifier reads the signing domain + selector out of the email itself and checks the sender's own DKIM signature against that domain's DNS key (validated against real FNB `d=fnbstatements.co.za` and Xero `d=post.xero.com` mail — genuine passes, every tampering rejected). No integration, no per-bank deal, no aggregator KYC. Everything runs on the applicant's device (WebCrypto + browser DNS-over-HTTPS for the public key); the only thing that ever leaves is a serverless share link — the redacted proof base64url-encoded in the URL fragment (never sent to any server) — so there is no storage and no honeypot. Honest trust tiers (this IS the product — never a "verified" badge a weaker proof can wear): - SOURCE-VERIFIED — a cryptographic anchor ties the document to a NAMED sender: a DMARC-aligned DKIM email signature, a signed PDF (PAdES), or a zkTLS session. The certificate may say "genuinely from ". - SELF-ATTESTED — sealed and tamper-evident since the user shared it, and only the chosen fields are revealed, but the SOURCE is not cryptographically verified: plain uploads, WhatsApp/Telegram forwards, portal downloads, screenshots. DKIM is channel-borne (lost when a file is forwarded); PAdES is file-borne (survives any channel); zkTLS is a re-fetch. The two tiers render as unmistakably different objects so a self-attested proof can never be mistaken for a source-verified one. Selective disclosure is SHA-256 Merkle inclusion proofs: reveal chosen fields, prove each is genuinely in the sealed statement, hide the rest, and prove nothing was added or removed — validated end-to-end (legit verifies; tamper, invented-row, wrong-key and inflate all rejected; hidden fields never present in the link). Trustless redaction of figures that appear in the email BODY is done with zk-email, and it is SHIPPED: a zero-knowledge circuit (built on zk.email's audited RSA/SHA circuits) verifies the sender's DKIM RSA-2048 signature and the body hash INSIDE the proof and reveals a single chosen figure — no notary, the whole document never travels. Proving runs on a private, prove-and-discard service (the email is seen only to prove it, then discarded, never stored); the recipient VERIFIES the ~KB proof on their own device, and the sender is NAMED trustlessly by matching the proof's key-hash to that domain's live DKIM key. Validated end-to-end: a real proof reveals exactly the chosen figure and nothing else, a forged figure is rejected, a tampered proof is rejected. Structured bank-app/API data (where the figure is only inside a PDF, e.g. a bank statement) routes to zkTLS/MPC-TLS — pull the value from the bank's authenticated session, no PDF, no aggregator. Honest scope: fully-trustless redaction of a field buried inside a PDF ATTACHMENT is an open problem (it would require parsing the PDF inside a ZK circuit, which is impractical today); whole-document authenticity is trustless and universal right now. Share a proof via the native share sheet, a copy link, or a QR code. ## Integration recipes -------------------------------------------------------------------------------- ### DuckDB — SQL straight over a compressed .at1 LOAD 'at1'; SELECT symbol, count(*), avg(price) FROM read_at1('trades-2026-01.at1') WHERE ts BETWEEN 1704067200000 AND 1704153600000 -- zone-maps skip non-matching row-groups GROUP BY symbol; ### SQLite — virtual table .load ./at1_vtab sqlite3_at1_init CREATE VIRTUAL TABLE trades USING at1('ticks.at1', 'agg_id','price','qty','ts_ms'); SELECT count(*), avg(price) FROM trades WHERE agg_id BETWEEN 100 AND 200; ### pandas / Polars / Dask — via Apache Arrow import at1_arrow df = at1_arrow.to_pandas("trades.at1") # whole table -> pandas sub = at1_arrow.to_polars("trades.at1", columns=["agg_id","price"], where={"agg_id": (100, 200)}) # pushed-down slice -> Polars at1_arrow.write_ipc("trades.at1", "trades.arrow") # Arrow IPC -> any engine # Dask: dd.from_pandas(at1_arrow.to_pandas("trades.at1"), npartitions=8) ### Postgres — foreign data wrapper (then ordinary SQL, including JOINs) CREATE EXTENSION at1_fdw; CREATE SERVER at1 FOREIGN DATA WRAPPER at1_fdw; CREATE FOREIGN TABLE trades ( agg_id bigint, price float8, qty float8, ts_ms bigint ) SERVER at1 OPTIONS (filename '/data/ticks.at1'); SELECT count(*), avg(price) FROM trades WHERE agg_id BETWEEN 100 AND 200; ### Trino / Presto / Spark / Flink — federation These engines reach AT-1 through the same C core via the Postgres FDW or a JDBC connector: Trino / Presto / Flink / Spark --(Postgres or JDBC)--> at1-postgres --at1_fdw--> at1_block.c --> trades.at1 ### S3-compatible storage — compress on PUT, decompress on GET (transparent) AT1_CLOUD_TOKEN=secret python at1_cloud.py serve ./store --bucket at1 \ --access-key AK --secret-key SK --port 9100 # any S3 SDK works (boto3, aws cli, DuckDB httpfs, Spark). Plaintext is NEVER stored: aws --endpoint-url http://localhost:9100 s3 cp events.csv s3://at1/data/events.csv # -> only a verified .at1 is stored; a failed round-trip returns HTTP 422 # response headers: x-at1-original-bytes, x-at1-compressed-bytes, x-at1-ratio aws --endpoint-url http://localhost:9100 s3 cp s3://at1/data/events.csv ./back.csv # -> transparent decompress; back.csv is byte-identical (HTTP Range supported) # SQL REST endpoint over the stored objects (reads only touched blocks): curl -s http://localhost:9100/sql -H "Authorization: Bearer secret" \ -d '{"sql": "SELECT id, user FROM data/events.csv WHERE score BETWEEN 10 AND 12 LIMIT 100"}' ### Auto-tier a directory to cold .at1 (with a tamper-evident ledger) at1-watch /data/archive --older-than 7d --include "*.csv,*.log,*.ndjson" --verify-ledger # optional: --delete-original (only after a verified round-trip), --dry-run, --once ### Estimate savings before committing (samples files, extrapolates totals) at1-doctor scan /data --report savings.html # prints per-file measured ratio + projected GB and $/yr at an assumed storage rate ### Ingest straight from a URL (plaintext never lands on disk) at1 fetch https://data.example.com/events.csv events.at1 columnar at1 integrity events.at1 # SHA-256 trailer: decode == original ### Front-end: React + TanStack Virtual (millions of rows, no backend) AT-1 stores tables as compressed ROW-GROUPS, which map 1:1 onto a virtualizer's visible window. The in-browser query WASM (at1_block.js/.wasm, the build behind /try) decodes one row-group on demand, so TanStack Virtual renders millions of rows from a single static .at1 with no API server and <1% of the file read. npm install @tanstack/react-virtual # + serve at1_block.js/.wasm from /public/wasm // open the file, read schema, decode the owning row-group per visible row (cached): // at1q_open -> at1q_total_rows / at1q_nrowgroups / at1q_rows_in_group / at1q_coltype // at1q_decode_int(handle, group, col, buf, cap) // integer columns // useVirtualizer({ count: totalRows, ... }); getRow(i) decodes only the visible group. // Pure-browser path decodes INTEGER columns; for text/mixed or remote files, drive a // windowed SQL query (SELECT * ... WHERE id BETWEEN start AND end) from the visible range. // Encode for WASM decode: at1 compress qcolumnar data.csv out.at1 --backend zstd --block-backend zstd --keep-queryable Full guide: https://tinyfiles.io/docs/tanstack-virtual ### Front-end: React + TanStack DB & Query (reactive collection from a compressed file) Hydrate a reactive TanStack DB collection (or a plain TanStack Query queryFn) straight from a compressed, queryable .at1. Full rows via the /sql endpoint (any column type, only touched blocks read), or numeric columns client-side via the WASM (no backend). After a one-time hydrate, useLiveQuery recomputes filters/sorts/joins on the client. npm install @tanstack/react-db @tanstack/query-db-collection @tanstack/query-core // queryFn loads rows from AT-1's /sql: SELECT * FROM data/trades.csv WHERE // createCollection(queryCollectionOptions({ queryKey, queryFn, queryClient, getKey })) // useLiveQuery(q => q.from({ t: coll }).where(...).orderBy(...)) -> reactive view // compose with TanStack Virtual to render the result. Honest scope: read/query collections, // not a high-frequency optimistic-write sync backend. Full guide: https://tinyfiles.io/docs/tanstack-db ### AI agents (Claude / Cursor / VS Code) via MCP # MCP server config: { "mcpServers": { "at1": { "command": "at1", "args": ["mcp"] } } } # the agent can then compress, decompress, query, and verify .at1 files as tools -------------------------------------------------------------------------------- ## Decode in your application (every binding returns the EXACT original bytes) -------------------------------------------------------------------------------- # Python from at1decode import decode original = decode(open("file.at1", "rb").read()) # JavaScript (browser / Node) import { decode } from "@tinyfiles/decoder"; const original = await decode(at1Bytes); # Go (cgo), Rust (FFI), and a portable C ABI are also provided; the WASM build # decodes in the browser with no server. See https://tinyfiles.io/docs/sdk -------------------------------------------------------------------------------- ## Images (a common first question) -------------------------------------------------------------------------------- AT-1 is lossless, so the result depends entirely on the input: - Raw / uncompressed (BMP, RAW, uncompressed TIFF, FITS), and DICOM medical images: REAL WINS (dedicated DICOM codec; byte-plane/delta transforms on raw pixels). - Screen captures / charts / synthetic: often a win. - Already-compressed JPEG / PNG / WebP / GIF / HEIC: ~ties — already entropy-coded; AT-1 keeps them byte-exact in a verified, addressable container (value = integrity, not ratio). AT-1 is NOT a smaller-than-JPEG lossy replacement. Compress: `at1 compress auto photo.bmp photo.at1` or `at1 compress dicom scan.dcm scan.at1`. Searchability: image pixels are NOT query-searchable. Keep a `qcolumnar` metadata sidecar (path, timestamp, label, boxes) and query THAT in place; for video, the media container supports find-the-moment + exact-frame extraction. -------------------------------------------------------------------------------- ## Common gotchas -------------------------------------------------------------------------------- - "not queryable": you compressed with a non-queryable codec. Use `qcolumnar`/`qjson`. - `qcolumnar` columns are positional with names from the CSV header row — keep the header; filter on integer/timestamp keys (decimal-range predicates are being refined). - Already-compressed inputs (.jpg/.mp4/.zip/.gz) won't shrink — expected, not a bug. - `card_required` / `payment_required`: a metering gate, not a crash; add a card or settle the balance. Decoding is never blocked. -------------------------------------------------------------------------------- ## Honest scope (we publish ties and losses, not just wins) -------------------------------------------------------------------------------- AT-1 wins on structured/lossless data and on the queryable + verified layer. It does NOT beat general compressors on natural images/video (an information-theory wall) and does NOT predict market price direction (tested exhaustively). Benchmarks on real data: https://tinyfiles.io/comparison -------------------------------------------------------------------------------- ## Links -------------------------------------------------------------------------------- Start here (plain English): https://tinyfiles.io/docs/start-here Examples & integrations: https://tinyfiles.io/docs/examples Query from your engine: https://tinyfiles.io/docs/engines Managed cloud / S3: https://tinyfiles.io/docs/cloud Images: https://tinyfiles.io/docs/images SDK & bindings: https://tinyfiles.io/docs/sdk CLI reference: https://tinyfiles.io/docs/cli Docs home: https://tinyfiles.io/docs Pricing: https://tinyfiles.io/pricing - [AT-1 lakehouse cold tier](https://tinyfiles.io/lakehouse): Store Delta/Iceberg/Parquet ~27% smaller than Parquet+zstd, byte-for-byte lossless, and query it in place. Verified byte-identical on Snowflake and Databricks against real NYC-TLC; read directly by Spark, Trino and DuckDB through the gateway. Saves storage not compute; reading is free, only tiering (writing) is metered. Does NOT work against Cloudflare R2 (Snowflake region defect). - [AT-1 Reveal](https://tinyfiles.io/reveal): Selective disclosure over video and documents. Withhold frames, blur a region, or hide rows, and the recipient can still prove every part they can see is byte-identical to the sealed original and that nothing was added or removed. Merkle root over parts; withheld parts leave their commitments behind. Verified client-side with no install or account. Does NOT judge whether the right things were redacted.