Feature extraction / compute-on-compressed — at1 features

An ML pipeline pays to decompress the same columns thousands of times. at1 featuresstores columns as block-compressed segments with a per-block zone map, then extracts features — counts, sums, means, min/max, and range-filtered aggregates — touching only the blocks whose zone map overlaps the predicate. Results are byte-identical to a full scan; a selective query reads a fraction of the bytes.

Use it

# build a block-compressed feature container (per-block zone maps)
at1 features build ticks.csv -o ticks.at1feat

# extract features over a range predicate — only blocks whose zone map overlaps are read
at1 features extract ticks.at1feat --where ts:1700262278:1700265482
#   WHERE ts in [...] (touched 1/128 blocks, read 4,559 of 585,272 bytes = 128.4x less):
#     qty: count=1601 sum=397013 mean=247.98 min=1 max=500

# no predicate -> summary features computed from zone maps alone, 0 bytes decoded
at1 features extract ticks.at1feat

What it gives you

  • Zone-map pushdown — blocks whose min/max can't overlap the filter are skipped without a single byte read; the reduction scales with how clustered your predicate is (up to 128× fewer bytes on the reference clustered predicate).
  • Exact, and verified — features are byte-identical to a full decompress-and-scan; every decoded block is SHA-checked against the manifest, so tampering is refused.
  • Free summaries — summary features come straight from the zone maps with 0 bytes decoded.

Honest scope

The reduction depends on physical clustering. A random-scattered predicate degrades toward a full scan — and the tool reports the exact bytes touched, so you always see where you sit rather than trusting a promised ratio.

See the marketing overview at AT-1 Features, and zero-scan GenQuery for the query-layer counterpart.

← CLI reference