D01 / Territory 1 of 49
Market Data Engineering
Build trustworthy price, quote, trade, and reference histories.
Published work
Released builds
in this territory.
Every card below is included in the current governed release with its article, sources, Python, TypeScript, and available labs.
Search the released libraryBar Construction
7 buildsMarket Data Engineering / Bar Construction
Time Bars
Groups a trade tape into fixed-duration bars. This is the sampling scheme every chart you have seen uses, and it is a *choice*: it samples the market at a constant rate regardless of how much is happening in it.Market Data Engineering / Bar Construction
Tick Bars
Closes a bar every N trades rather than every N seconds. Bars then arrive at the rate the market is transacting, so quiet periods produce fewer of them.Market Data Engineering / Bar Construction
Volume Bars
Closes a bar once a target share volume has traded. Sampling by volume rather than by clock gives series with far more stable statistical properties than time bars.Market Data Engineering / Bar Construction
Dollar Bars
Closes a bar once a target traded *value* is reached. Unlike volume bars this stays comparable as the price level changes — 1,000 shares of a $10 stock and of a $500 stock are not the same event.Market Data Engineering / Bar Construction
Tick-Imbalance Bars
Closes a bar when signed tick flow becomes unusually one-sided relative to what recent history led you to expect.Market Data Engineering / Bar Construction
Volume-Imbalance Bars
The imbalance rule applied to signed *volume* rather than signed tick count, so one large order weighs more than many small ones pointing the same way.Market Data Engineering / Bar Construction
Tick-Run Bars
Closes a bar when a run of same-signed trades exceeds what the recent buy probability makes plausible.Cleaning and Validation
6 buildsMarket Data Engineering / Cleaning and Validation
OHLC Consistency Validator
Checks each bar against the invariants an OHLC bar must satisfy — high is the maximum, low is the minimum, open and close lie between them — with a tick-size tolerance so representable rounding is not reported as corruption.Market Data Engineering / Cleaning and Validation
Hampel Bad-Tick Filter
Flags points that sit too far from a rolling median, measured in robust deviations rather than standard deviations — so one fat-finger print cannot inflate the very statistic used to detect it.Market Data Engineering / Cleaning and Validation
Median Absolute Deviation Outlier Filter
Whole-sample outlier detection against the median absolute deviation.Market Data Engineering / Cleaning and Validation
Stale-Quote Detector
Separates the several distinct ways a quote can be stale: old at the source, delayed in transport, unchanged for too long, or arriving after a missed heartbeat.Market Data Engineering / Cleaning and Validation
Duplicate-Trade Resolver
Reconciles a stream of new, corrected and cancelled trade messages into one authoritative set.Market Data Engineering / Cleaning and Validation
Crossed/Locked Market Detector
Classifies a quote as normal, locked (bid equals ask) or crossed (bid above ask).Time Synchronization
5 buildsMarket Data Engineering / Time Synchronization
Previous-Tick Interpolation
Samples a series onto a grid by carrying the last value known *at that moment* forward.Market Data Engineering / Time Synchronization
Linear Quote Interpolation
Interpolates between the quotes either side of a target time. Not causal, and no accuracy advantage over carrying the previous quote forward is claimed.Market Data Engineering / Time Synchronization
Refresh-Time Sampling
Builds a common clock for several instruments by advancing only when every one of them has refreshed. One established construction, not the standard remedy.Market Data Engineering / Time Synchronization
Exchange-Calendar Alignment
Validates a trading-calendar bundle before anything is aligned to it: that sessions do not overlap, that closures are consistent with sessions, and that the licence terms permitting redistribution are present.Market Data Engineering / Time Synchronization
Asynchronous Return Alignment
Compares two return intervals and reports exactly how they overlap.Data Quality
6 buildsMarket Data Engineering / Data Quality
Missing-Bar Gap Classifier
Decides why a bar is missing: the session was closed, the instrument was halted, the feed dropped, or a sequence number was skipped.Market Data Engineering / Data Quality
Feed-Latency Monitor
Validates that a latency measurement is meaningful before any latency is computed from it.Market Data Engineering / Data Quality
Price-Source Consensus Check
Reconciles quotes from several providers into one defensible price, or refuses to.Market Data Engineering / Data Quality
Schema-Drift Detector: Catch Structural and Semantic Contract Changes
Compares two versions of a feed schema and classifies every change as compatible or breaking.Market Data Engineering / Data Quality
Point-in-Time Availability Guard
Reconstructs what was actually knowable at a given moment, using each record's available_at rather than its observation_time.Market Data Engineering / Data Quality
Provider Adjustment-Basis Drift Detector
Compares archived provider adjustment factors and isolates unexplained residual drift after newly knowable corporate actions.Order-Book Feed Engineering
7 buildsMarket Data Engineering / Order-Book Feed Engineering
Trade-and-Quote Event Normalization
Normalises raw venue trade and quote messages into one canonical event shape.Market Data Engineering / Order-Book Feed Engineering
Level-2 Snapshot-and-Delta Reconstruction
Rebuilds an aggregated price-level book from a snapshot plus the deltas that follow it.Market Data Engineering / Order-Book Feed Engineering
Level-3 Order-by-Order Reconstruction
Rebuilds the book order by order rather than by price level. Each visible order keeps its identity and quantity; the emitted priority_rank is insertion order within the reconstructed stream, not a venue queue position.Market Data Engineering / Order-Book Feed Engineering
Sequence-Gap Detection and Recovery
Detects missing sequence numbers and decides whether the stream can continue or must resynchronise from a snapshot.Market Data Engineering / Order-Book Feed Engineering
Price-Level Quantity Aggregation
Collapses individual orders into quantity per price level — the Level 3 to Level 2 projection, and the form most analytics actually consume.Market Data Engineering / Order-Book Feed Engineering
Snapshot/Incremental-Feed Reconciliation
Reconciles a periodic snapshot against the book built from incrementals.Market Data Engineering / Order-Book Feed Engineering
Multi-Venue Best-Quote and Book Consolidation
Merges books from several venues into one consolidated view and identifies the best bid and offer across them.Territory coverage
What this domain
is mapped to cover.
The published builds above are the released slice of this map. Every subject below belongs to this territory's charter.