How to Store Market Data PCAPs: Split, Compress
Retention arithmetic for a 2 TB per day feed at three compression ratios (illustrative inputs)series ·
2026-09-17 · 3×4
Retention arithmetic for a 2 TB per day feed at three compression ratios (illustrative inputs)
Retention arithmetic for a 2 TB per day feed at three compression ratios (illustrative inputs)
| scenario | gb_per_day | tb_per_year | tb_two_years |
|---|---|---|---|
| ratio 3 | 667 | 166.7 | 333.3 |
| ratio 5 | 400 | 100 | 200 |
| ratio 8 | 250 | 62.5 | 125 |
the exact SQL behind every number
SELECT
concat('ratio ', toString(ratio)) AS scenario,
round(raw_tb_per_day * 1000 / ratio) AS gb_per_day,
round(raw_tb_per_day * sessions / ratio, 1) AS tb_per_year,
round(raw_tb_per_day * sessions * 2 / ratio, 1) AS tb_two_years
FROM
(
SELECT
2 AS raw_tb_per_day,
250 AS sessions,
arrayJoin([3, 5, 8]) AS ratio
)
ORDER BY ratio
Related queries
AAPL trade counts and distinct prices per quarter hour
series 64×3
→
Uncompressed footprint of five names as retention grows
ranking 6×2
→
Distinct prices per name, and how often each one repeats
ranking 5×3
→
One session of trade prints, five household names
ranking 5×3
→
See all 3,293 queries →