The governed definitions this build depends on. Read them first if a term is unfamiliar.
- PrimaryData ReconciliationData reconciliation is the controlled comparison of records or aggregates across sources or processing stages to explain differences.
- PrerequisiteSeries IdentitySeries identity is the complete key and semantic contract that states what a sequence of observations represents.
- ImportantData LineageData lineage records where data came from, how it was transformed, and which entities, activities, and agents affected it.
- ImportantInstrument CurrencyInstrument currency is the currency in which a financial instrument's contractual amounts or reference terms are denominated.
- ImportantMarket Data FeedA market data feed is a source-defined stream or delivery product carrying market events, reference data, or aggregates.
- ImportantMeasurement UnitA measurement unit states the quantity scale represented by a numeric value, such as currency, shares, contracts, seconds, or percent.
- ImportantNumerical ToleranceNumerical tolerance is a declared bound used to decide when two numeric results are acceptably close for a specific comparison.
- ImportantOutlierAn outlier is an observation that is unusually distant or inconsistent under a specified comparison model.
- ImportantPrecisionPrecision describes the fineness or number of digits with which a value or timestamp is represented or computed.
- ImportantSnapshotA snapshot is a bounded representation of selected market or reference-data state at a stated cutoff.
- ImportantStale DataStale data is information whose age or lack of expected update makes it unsuitable under a declared freshness policy.
Practical promise. Decide whether enough fresh, contract-compatible, independently owned sources agree - and return no price when the evidence does not support one.
Four feeds showing the same number may still be one observation copied four times. Four fresh prices may still be incomparable if one is an adjusted close, another is a midpoint, and two are last trades. The useful question is not "What is the median of these numbers?" It is:
Which observations are eligible, which independent owners agree, and does quorum survive the disagreement check?
This is a data-quality diagnostic. It does not estimate fair value, prefer a venue, repair bad inputs, or prove a market price is correct.
Eligibility comes before statistics
Every candidate row must match the same instrument, currency, price type, adjustment convention, and session. Its event time must be no later than the decision time and no older than the configured freshness limit. Its owner must also be unique in the snapshot.
The owner rule is intentionally conservative: if two labels map to one owner,
both are excluded as shared_owner. Choosing one of them silently would add a
source-selection policy to a consensus algorithm. Production systems should
maintain ownership and upstream lineage in a governed registry; names alone do
not prove independence.
FIX market-data definitions distinguish price, currency, entry type, time, market, and session fields, while Nasdaq ITCH defines price precision and event timestamps. These are examples of why a price has semantics, not just a numeric value. The package applies an exact-match policy and never performs implicit conversion or normalization.
Three outcomes, not one forced number
| Outcome | Meaning | Price output |
|---|---|---|
consensus | Eligibility quorum and inlier quorum both hold. | Median of inliers. |
insufficient_sources | Eligibility leaves too few independent owners. | null |
disputed | Dispersion is too broad, or removing outliers breaks quorum. | null |
That distinction preserves the reason a value is unavailable. A stale-source outage is operationally different from two fresh clusters that disagree.
The takeaway is simple: availability, agreement, and final value are separate decisions.
The robust band
For eligible prices , compute:
NIST defines MAD and the 0.6745 normal-consistent scaling. The scaling gives
standard-deviation-like units under a normal model; it does not assert that
source prices are normally distributed.
The package then applies explicit engineering policy:
where is an absolute floor and is a multiplier. A price is inside when . The numerical epsilon preserves an inclusive boundary under binary floating-point arithmetic.
The policy also sets a maximum allowed tolerance . If , the result
is disputed. That cap prevents an inflated MAD from widening the band until
incompatible clusters appear to agree. NIST discusses masking as a general
multiple-outlier risk; the cap itself is this package's conservative control,
not a universal statistical rule.
Zero MAD is a defined branch
If at least half the eligible prices equal the median, MAD can be zero. The algorithm does not divide by zero and does not borrow scale from an earlier snapshot. It sets scaled MAD and adaptive tolerance to zero, so the applied tolerance is exactly the absolute floor.
- With a floor of
0, only prices equal to the median are inliers. - With a floor of
0.03, a price exactly0.03away is included. - The floor is a currency-unit policy, so it must be reviewed when price scale or currency changes.
This behavior is deterministic in both implementations and has a dedicated boundary test.
Worked example: one isolated error
Suppose four eligible, independently owned sources report:
| Source | Price | Distance from initial median |
|---|---|---|
| A | 100.001 | 0.0105 |
| B | 100.009 | 0.0025 |
| C | 100.014 | 0.0025 |
| D | 100.380 | 0.3685 |
The median is 100.0115 and MAD is 0.0065. Scaled MAD is about 0.00964.
With k=3.5 and an absolute floor of 0.03, the applied tolerance is about
0.03373. A, B, and C remain inliers, D is named as an outlier, quorum of three
holds, and the final inlier median is 100.009.
The static diagram uses distinct shapes and labels as well as color, so source membership remains understandable without animation.
Worked example: conflict rather than false consensus
Now let three owners report 100.00 and three report 101.00. The median is
100.50 and MAD is 0.50. With k=3.5, adaptive tolerance is about 2.5945 - far
above the configured maximum of 0.15. The result is disputed, with no
consensus price.
That is not a claim that either cluster is wrong. It is an admission that this check cannot justify choosing one.
Use the playground as an audit instrument
Open the interactive consensus lab. It contains 24 synthetic snapshots across four scenarios:
- a clean control with an isolated erroneous price;
- zero-MAD and inclusive-floor boundaries;
- stale, mismatched, and commonly owned inputs;
- split clusters and post-filter quorum loss.
Choose a scenario, then use Back, Step, Play, Pause, and Reset. Change the multiplier, floor, maximum tolerance, or freshness limit and observe all dependent outputs recompute. The initial screen is already informative; no interaction is required to discover the lesson. Reduced-motion preference turns Play into a single deterministic step.
Implementation and verification
Python and TypeScript implement the same sequence:
- validate policy, UTC timestamps, and unique source IDs;
- exclude duplicated owners, invalid prices, semantic mismatches, future events, and stale events;
- return
insufficient_sourcesif eligibility quorum fails; - calculate median, MAD, floor, and the maximum-tolerance gate;
- partition sources with an inclusive boundary;
- return
disputedif inlier quorum fails; otherwise return the inlier median.
The shared operational fixture contains 24 deterministic decisions. Tests also cover duplicate IDs, non-UTC timestamps, invalid policy, future events, exact boundary equality, source ownership, semantic mismatch, broad conflict, and post-filter quorum loss. Passing them proves implementation parity, not market accuracy or investment value.
What can still go wrong
- Ownership metadata can be incomplete: separately branded vendors may share one upstream feed.
- Several independent systems can apply the same wrong split adjustment.
- A synchronized market repricing can be correct even when a stale consumer labels it an outlier.
- The floor and maximum tolerance are expressed in price units and do not transfer automatically across currencies or price scales.
- A last trade, midpoint, auction price, and adjusted close are different contracts even when their numbers are close.
- A price consensus is not a best-execution, benchmark, valuation, or trading recommendation.
Keep raw records, registry versions, exclusions, parameters, and the complete diagnostic output. Do not overwrite an observation merely because the check disagrees with it.
Why there is no historical provider example yet
This revision intentionally does not present a real provider disagreement. A credible example needs contemporaneous raw snapshots from at least three independently owned sources, redistributable licenses, exact price semantics, clock lineage, ownership as known at that date, and a later authoritative correction or explanation. A single vendor API cannot demonstrate cross-source consensus, and screenshots gathered at different times cannot prove a simultaneous discrepancy.
Until that evidence can be archived and redistributed, a named historical story would be anecdotal. The package therefore labels its 24 cases as synthetic mechanics tests and records historical-case acceptance criteria in REFERENCES.md.
Evidence ledger
| Kind | What this article uses |
|---|---|
| Sourced fact | NIST MAD/scaling and masking discussion; Nasdaq and FIX field semantics. |
| Provider observation | Official protocols attach precision, time, type, currency, market, and session context to prices. |
| Derivation | Arithmetic, statuses, and diagrams computed from the CC0 synthetic fixtures. |
| Interpretation | Eligibility precedes quorum; broad dispersion is conflict, not automatic agreement. |
Summary and next topic
A defensible consensus begins before the median. First establish comparable contracts, freshness, and independent ownership. Then apply the robust band, make zero-MAD behavior explicit, cap excessive dispersion, and recheck quorum. When evidence is insufficient or disputed, return no price.
Continue with Schema-Drift Detector to protect the input contract that this check depends on.
Primary references and visual assets
See REFERENCES.md for source roles, versions, limitations, and historical-case criteria.
| Asset | Teaching role | Source | Status |
|---|---|---|---|
| Decision map | Eligibility, quorum, and conflict states | ../visuals/static/article-hero.svg | Ready |
| Worked band | Exact isolated-error arithmetic | ../visuals/static/consensus-band.svg | Ready |
| Interactive lab | Scenario and parameter sensitivity | ../visuals/animated/price-consensus-playground.html | Ready |
References4 primary sources and evidence notesExpand the source trail, evidence role, and limitations behind the engineering choices.
Expand the source trail, evidence role, and limitations behind the engineering choices.
R01 - Median Absolute Deviation
- Organization or authors: National Institute of Standards and Technology (NIST)
- Source type: Official statistical reference
- Publication or effective date: 2016-04-11
- Version: Dataplot reference page
- URL or DOI: https://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/mad.htm
- Accessed: 2026-07-22
- Jurisdiction: General statistical method
- Supports: MAD definition and
MAD / 0.6745normal-consistent scaling. - Limitations: Does not prescribe source eligibility, quorum, market-data tolerances, or a maximum band.
R02 - Detection of Outliers
- Organization or authors: NIST/SEMATECH
- Source type: Official engineering statistics handbook
- Publication or effective date: Living handbook
- Version: Section 1.3.5.17, accessed 2026-07-22
- URL or DOI: https://www.itl.nist.gov/div898/handbook/eda/section3/eda35h.htm
- Accessed: 2026-07-22
- Jurisdiction: General statistical method
- Supports: Modified-z-score context and the warning that multiple outliers can produce masking.
- Limitations: The package's
maximum_toleranceis an engineering control motivated by this risk, not a NIST-prescribed market-data rule.
R03 - Nasdaq TotalView-ITCH 5.0
- Organization or authors: Nasdaq
- Source type: Official exchange data-protocol specification
- Publication or effective date: Revision log current through 2023-04-28 in the retrieved document
- Version: 5.0
- URL or DOI: https://classic.nasdaqtrader.com/content/technicalsupport/specifications/dataproducts/NQTVITCHSpecification.pdf
- Accessed: 2026-07-22
- Jurisdiction: United States, Nasdaq market data
- Supports: Price fields use declared fixed-point precision; timestamps are nanoseconds since midnight; feed messages carry event-specific context.
- Limitations: Does not define cross-provider ownership, staleness, or this consensus method.
R04 - FIX MarketDataSnapshotFullRefresh
- Organization or authors: FIX Trading Community
- Source type: Official protocol repository
- Publication or effective date: FIX 5.0 repository page
- Version: FIX 5.0 message definition, accessed 2026-07-22
- URL or DOI: https://fiximate.fixtrading.org/legacy/en/FIX.5.0/body_514887.html
- Accessed: 2026-07-22
- Jurisdiction: Protocol standard
- Supports: Market-data entries distinguish price, entry type, currency, date/time, market, session, quote condition, and trade condition.
- Limitations: The fields motivate semantic gates but do not imply that unlike entries should be normalized or averaged.
Evidence-role ledger
| Statement type | Evidence in this package |
|---|---|
| Sourced fact | R01 defines MAD/scaling; R02 discusses masking; R03/R04 document protocol fields. |
| Provider observation | R03 and R04 show that price values arrive with precision, time, type, currency, market, and session context. |
| Package derivation | Worked arithmetic and all 24 scenario outputs derive from CC0 synthetic fixtures and tested code. |
| Interpretation | Stale, mismatched, or commonly owned rows must not manufacture agreement; broad dispersion is reported as conflict. |
Historical-case acceptance criteria
No historical market observation is claimed. A future case must archive or identify all of the following before publication:
- contemporaneous raw observations from at least three independently owned sources;
- licenses that permit the quoted values and timestamps to be redistributed;
- instrument, venue, currency, price-type, adjustment, and session identity;
- event, receive, and availability timestamps with clock provenance;
- source ownership and upstream lineage as known at the event date;
- an authoritative correction, status notice, or source-side explanation;
- a clear separation among captured fact, algorithmic derivation, and analyst interpretation.
A single-provider API, a chart screenshot, or values collected on different dates cannot establish a historical cross-source consensus event.
Full dependency-light reference implementations in both supported languages.
export const NORMAL_MAD_DENOMINATOR = 0.6745;
const CONTRACT_FIELDS = ["instrument_id", "currency", "price_type", "adjustment", "session"] as const;
export interface SourceQuote {
source_id: string;
owner_id: string;
price: number;
instrument_id: string;
currency: string;
price_type: string;
adjustment: string;
session: string;
event_time: string;
}
export interface Snapshot { as_of: string; quotes: SourceQuote[] }
export interface ConsensusPolicy {
minimum_independent_sources: number;
z_threshold: number;
absolute_tolerance: number;
maximum_tolerance: number;
max_age_ms: number;
expected_contract: Record<(typeof CONTRACT_FIELDS)[number], string>;
}
function median(values: number[]): number {
const ordered = [...values].sort((a, b) => a - b);
const middle = Math.floor(ordered.length / 2);
return ordered.length % 2 ? ordered[middle] : (ordered[middle - 1] + ordered[middle]) / 2;
}
function utcMillis(value: unknown): number {
if (typeof value !== "string" || !value.endsWith("Z")) throw new Error("timestamps must be UTC strings ending in Z");
const parsed = Date.parse(value);
if (!Number.isFinite(parsed)) throw new Error("timestamps must be valid UTC instants");
return parsed;
}
export function consensus(snapshot: Snapshot, policy: ConsensusPolicy) {
const { minimum_independent_sources: minimum, z_threshold: z, absolute_tolerance: floor,
maximum_tolerance: maximum, max_age_ms: maxAge, expected_contract: contract } = policy;
if (!Number.isInteger(minimum) || minimum < 2 || !Number.isFinite(z) || z <= 0 ||
!Number.isFinite(floor) || floor < 0 || !Number.isFinite(maximum) || maximum < floor ||
!Number.isInteger(maxAge) || maxAge < 0 || !contract || CONTRACT_FIELDS.some((field) => !(field in contract))) {
throw new Error("invalid consensus policy");
}
if (!snapshot || !Array.isArray(snapshot.quotes)) throw new Error("snapshot.quotes must be a list");
const asOf = utcMillis(snapshot.as_of);
const ids = snapshot.quotes.map((quote) => quote?.source_id);
if (ids.some((id) => typeof id !== "string" || id.length === 0)) throw new Error("every quote needs a non-empty source_id");
if (new Set(ids).size !== ids.length) throw new Error("source_id values must be unique within a snapshot");
const owners = new Map<string, number>();
for (const quote of snapshot.quotes) owners.set(quote.owner_id, (owners.get(quote.owner_id) ?? 0) + 1);
const eligible: SourceQuote[] = [];
const excluded: Array<{source: string; reason: string}> = [];
for (const quote of snapshot.quotes) {
let reason: string | null = null;
if (typeof quote.owner_id !== "string" || quote.owner_id.length === 0) reason = "missing_owner";
else if ((owners.get(quote.owner_id) ?? 0) > 1) reason = "shared_owner";
else if (!Number.isFinite(quote.price) || quote.price <= 0) reason = "invalid_price";
else {
const mismatch = CONTRACT_FIELDS.find((field) => quote[field] !== contract[field]);
if (mismatch) reason = `contract_mismatch:${mismatch}`;
else {
try {
const age = asOf - utcMillis(quote.event_time);
if (age < 0) reason = "future_event";
else if (age > maxAge) reason = "stale";
} catch { reason = "invalid_event_time"; }
}
}
if (reason) excluded.push({ source: quote.source_id, reason });
else eligible.push(quote);
}
const base = { observed_source_count: snapshot.quotes.length, eligible_independent_source_count: eligible.length,
minimum_independent_sources: minimum, excluded };
if (eligible.length < minimum) return { status: "insufficient_sources", reason: "quorum_not_met_after_eligibility",
consensus_price: null, inliers: [], outliers: [], diagnostics: base };
const center = median(eligible.map((quote) => quote.price));
const mad = median(eligible.map((quote) => Math.abs(quote.price - center)));
const robustScale = mad > 0 ? mad / NORMAL_MAD_DENOMINATOR : 0;
const adaptiveTolerance = z * robustScale;
const tolerance = Math.max(floor, adaptiveTolerance);
const diagnostics: Record<string, unknown> = { ...base, initial_center: center, mad, robust_scale: robustScale,
adaptive_tolerance: adaptiveTolerance, absolute_tolerance: floor, applied_tolerance: tolerance,
maximum_tolerance: maximum, zero_mad_policy: mad === 0 ? "absolute_floor" : "not_applicable" };
if (tolerance > maximum) return { status: "disputed", reason: "dispersion_exceeds_maximum_tolerance",
consensus_price: null, inliers: [], outliers: eligible.map((quote) => quote.source_id), diagnostics };
const comparisonEpsilon = 1e-12;
diagnostics.comparison_epsilon = comparisonEpsilon;
const inliers = eligible.filter((quote) => Math.abs(quote.price - center) <= tolerance + comparisonEpsilon);
const outliers = eligible.filter((quote) => Math.abs(quote.price - center) > tolerance + comparisonEpsilon);
diagnostics.inlier_independent_source_count = inliers.length;
if (inliers.length < minimum) return { status: "disputed", reason: "quorum_lost_after_outlier_filter",
consensus_price: null, inliers: inliers.map((quote) => quote.source_id),
outliers: outliers.map((quote) => quote.source_id), diagnostics };
return { status: "consensus", reason: "quorum_met", consensus_price: median(inliers.map((quote) => quote.price)),
inliers: inliers.map((quote) => quote.source_id), outliers: outliers.map((quote) => quote.source_id), diagnostics };
}
The embedded lab now expands to its full document height, keeping the article as the only scroll surface.