Hampel Bad-Tick Filter: Flag a Suspicious Print Without Looking Ahead

Key concepts

The governed definitions this build depends on. Read them first if a term is unfamiliar.

PrimaryOutlierAn outlier is an observation that is unusually distant or inconsistent under a specified comparison model.
ImportantMissing ValueA missing value indicates that the expected information is unavailable, unobserved, inapplicable, or withheld under a stated data contract.
ImportantNumerical ToleranceNumerical tolerance is a declared bound used to decide when two numeric results are acceptably close for a specific comparison.
ImportantTickA tick is one atomic market-data event or update in an ordered event stream.
ImportantTradeA trade is an executed transaction in which a quantity of a financial instrument changes hands at agreed terms.
MentionedImputationImputation is the governed estimation or substitution of values where observations are missing.

Practical promise. Build a local robust diagnostic that can run as each tick arrives, explain every decision, and keep a suspected anomaly separate from any correction policy.

Causal Hampel filtering contrasted with retrospective centered context

A price of 112 surrounded by prices near 100 is worth investigating. It is not automatically a bad tick. It might be an erroneous report, but it might also be a genuine repricing, an auction, a corporate-action discontinuity, a unit mistake upstream, or a valid trade with a special condition.

The Hampel diagnostic answers a narrower question: is the current value unusually far from a robust local center? This article makes that question safe for live use by using a trailing causal window by default. It also shows exactly why the familiar centered window belongs to retrospective analysis.

The supplied values are synthetic. They verify mechanics; they do not establish market facts, predictive value, or a profitable signal.

Keep three decisions separate

A production cleaning pipeline has at least three layers:

Eligibility: should this record participate at all, given its source, sale condition, unit, session, sequence, and correction status?
Diagnostic: does the declared Hampel rule flag it?
Policy: should an operator quarantine, annotate, replace, or leave it unchanged after corroborating evidence?

This package implements the diagnostic. It preserves raw values by default and emits a median replacement only as a suggestion. Calling a statistical outlier an exchange error skips the evidence layer.

Causal and centered windows are different algorithms

Let $i$ be the current index and $k$ a positive integer radius. The default causal window is:

W_i^{\text{causal}}=\{x_j:\max(0,i-k)\le j\le i\}

At full history it contains at most $k+1$ positions, including the current value. It never reads beyond $i$ .

The retrospective centered window is:

W_i^{\text{centered}}=\{x_j:\max(0,i-k)\le j\le\min(n-1,i+k)\}

An interior centered window contains $2k+1$ positions. The $k$ positions after $i$ are future information relative to a live decision.

Mode	Full window size	Future values?	Use
Causal, default	$k+1$	No	Live monitoring and reproducible replay
Centered	$2k+1$	Yes, except at the final boundary	Retrospective quality review

Boundary windows shrink instead of padding. Missing values remain in the sequence but are omitted from median and MAD calculations.

The robust local calculation

For the valid observations in $W_i$ , compute the median:

m_i=\operatorname{median}(W_i)

Then compute the unscaled median absolute deviation:

d_i=\operatorname{median}_{x\in W_i}|x-m_i|

This package uses the arithmetic mean of the two middle sorted values for an even-size median. That convention matters because a causal radius-3 window normally contains four values.

Scale the MAD and calculate the local score:

\widehat{\sigma}_i=c d_i

s_i=\frac{|x_i-m_i|}{\widehat{\sigma}_i}

Symbol	Meaning
$x_i$	Current finite value
$W_i$	Declared causal or centered window after omitting missing values
$m_i$	Window median
$d_i$	Unscaled window MAD
$c$	Positive scale factor; default 1.4826
$s_i$	Local robust score
$h$	Positive threshold; default 3

Flag only when $s_i>h$ . Equality is not a flag.

The default $c=1.4826$ is approximately $1/0.6745$ . It calibrates MAD toward standard deviation under a normal reference model. It does not assert that market ticks are normally distributed, and it does not make $h=3$ universally correct.

Work the causal example by hand

Consider the synthetic sequence around index 5:

Plain text

index:  2      3      4      5
value: 99.9  100.0  100.1  112.0

With $k=3$ , the causal window contains those four values. The middle pair after sorting is 100.0 and 100.1, so:

m_5=\frac{100.0+100.1}{2}=100.05

The absolute deviations are 0.15, 0.05, 0.05, and 11.95. Their middle pair is 0.05 and 0.15:

d_5=\frac{0.05+0.15}{2}=0.10

The score is:

s_5=\frac{11.95}{1.4826\times0.10}\approx80.60

Because $80.60>3$ , index 5 is flagged. The default output is still 112.0. The local median 100.05 is stored separately as a suggested replacement, not silently written over the record.

Causal four-value Hampel window with exact median, MAD, and score

Warm-up, missing values, and zero MAD

These policies are part of the algorithm, not implementation trivia.

Minimum history

The default min_history is three valid observations. Earlier points may expose a median and MAD for inspection, but their score is null, status is insufficient_history, and they cannot be flagged. A missing observation does not help satisfy the minimum.

Missing and non-finite values

None in Python and null in TypeScript represent missing values. They are omitted from neighboring window statistics and remain missing at their own output index. NaN, positive or negative infinity, booleans, and non-numeric values are rejected rather than coerced.

Zero MAD

A flat local window produces $d_i=0$ . Dividing by zero would make implementations drift, so the package declares a fallback:

if the current value equals the median, set $s_i=0$ and status zero_mad_match;
if it differs, set $s_i=+\infty$ and status zero_mad_deviation.

The infinite score is a deterministic alert, not proof of bad data. Flat and discrete markets can produce this state naturally.

Future values can reverse a decision

The sequence [100, 100, 100, 100, 101, 101, 101] exposes the difference.

At index 4, the causal radius-3 window is [100, 100, 100, 101]. Its median is 100 and MAD is zero, so 101 receives an infinite score and is flagged.

The centered radius-3 window also includes the two future 101 values. Its median becomes 100.5 and MAD becomes 0.5. The score falls below the threshold, so it is not flagged.

Neither calculation is hidden in the playground. The centered result carries lookahead_used=true whenever a future index participates.

Rendering system map…

The takeaway is simple: window selection affects the fact pattern available to the detector, while repair belongs after diagnosis and evidence.

Use the playground to compare knowledge states

Open the interactive Hampel causality lab.

The initial screen already displays the full synthetic scenario and the first eligible decision. Use:

Back and Step to move one observation at a time;
Play/Pause to call the same transition repeatedly;
Reset to restore the same scenario and index;
the scenario selector to compare the canonical spikes, the zero-MAD causality reversal, and a clustered-anomaly failure case; and
the mode comparison to inspect causal and centered medians, MADs, scores, flags, and future-use labels at the same index.

The clustered scenario demonstrates masking: several extreme observations can enter one another's windows and change the robust center or scale. Hampel filtering is robust to sparse contamination, not immune to arbitrary regimes.

Implementation contract

The Python and TypeScript functions share these defaults:

Plain text

window radius = 3
threshold = 3
scale = 1.4826
minimum history = 3
mode = causal
repair = none

Both return raw value, mode, inclusive window bounds, valid count, median, MAD, scaled MAD, score, strict flag, status, look-ahead label, suggested replacement, and policy-controlled output.

The reference implementation favors auditable sorting inside each window over a specialized streaming median data structure. Production optimization can reduce repeated sorting, but it must preserve all boundary and diagnostic semantics.

What the tests establish

The shared fixture drives both languages. Tests cover:

expected flags in causal and centered modes;
causal default and absence of look-ahead;
explicit centered look-ahead;
even and odd medians;
minimum-history boundaries;
missing-value omission and preservation;
zero-MAD match and deviation states;
a causal-versus-centered outcome reversal;
strict threshold equality;
separation of flagging and repair;
invalid modes, policies, parameters, and non-finite values; and
retention of the 140-row synthetic stress corpus.

Passing those tests means the implementations match the declared package. It does not mean the parameters are optimal for an instrument or that flags identify true exchange errors.

What would make a real historical example trustworthy?

A high-value case needs two linked evidence layers:

the exact disseminated print with source, timestamp, sequence, trade ID, conditions, and raw price; and
authoritative later evidence showing whether it was corrected, cancelled, or marked erroneous.

NYSE's current Daily TAQ specification documents fields for sequence, trade ID, source, sale conditions, and correction indicators. Those fields describe how a rigorous reconstruction could be made. The product data are licensed, and this package did not verify a legally redistributable raw-and-correction pair.

Therefore, no historical print is presented as fact here. A spike visible on one vendor is a provider outlier, not a confirmed bad tick. Even agreement among several providers could reflect shared upstream data rather than independent confirmation.

Evidence	Safe interpretation
Synthetic fixture plus tests	Verified algorithm mechanics
One provider's extreme price	Investigation candidate
Cross-source disagreement with identity and time	Confirmed discrepancy, unresolved cause
Raw print linked to official correction/cancel/error	Confirmed operational status of that record

This distinction prevents a plausible chart from becoming misinformation.

Production failure modes

Valid jumps: news, auctions, halts, corporate actions, or price-limit events can be real.
Wrong partition: mixing instruments, currencies, venues, sessions, or adjusted and unadjusted prices creates artificial anomalies.
Condition blindness: an eligible-price rule may depend on sale conditions outside this numerical filter.
Clusters and regimes: several anomalies can mask one another; a genuine level shift can generate repeated flags.
Irregular time: four indexes may span milliseconds or minutes.
Flat markets: zero MAD makes any first change an infinite-score event under this declared rule.
Silent replacement: overwriting the raw value destroys the evidence needed to review the detector.
Centered leakage: a smooth retrospective chart can falsely suggest a decision was available live.

Monitor flag rates, insufficient-history rates, zero-MAD states, source concentration, and later correction outcomes. Version the parameters and eligibility rules.

Summary

A reliable Hampel bad-tick diagnostic is more than median ± three MADs. It needs a declared knowledge boundary, median convention, scale, threshold equality, warm-up, missing-value behavior, zero-MAD fallback, and repair policy.

Use the causal trailing window for live claims. Label the centered version retrospective. Preserve every raw observation. Most importantly, call the result what it is: an anomaly flag awaiting evidence, not proof that the market printed a mistake.

Continue with Median Absolute Deviation Outlier Filter for robust scale outside a moving-window state, or Trade/Quote Condition-Code Filter for the eligibility layer that should precede numerical screening.

Primary references

Hampel's original influence-curve paper supplies the robust-estimation motivation.
NIST defines MAD, its normal-reference scaling, and masking cautions.
Pearson and coauthors describe standard and generalized Hampel filters.
NYSE Daily TAQ Client Specification v4.3 documents current trade ordering and correction-evidence fields.

Full source roles, dates, versions, and limitations are recorded in REFERENCES.md.

Workflows

Rendered from the canonical Mermaid sources linked by this article.

Hampel diagnostic, evidence, and policy flow

Purpose: show that the numerical flag is one stage in an auditable cleaning process.

Rendering system map…

Takeaway: a flag is a reproducible diagnostic; proof and repair require separate evidence and policy.

ReferencesPrimary sources and evidence notes

Expand the source trail, evidence role, and limitations behind the engineering choices.

Accessed 2026-07-22. The exact causal window, minimum-history, threshold, zero-MAD, missing-value, and repair rules are declared package choices, not defaults attributed to the sources.

HAMPEL-1974 — The Influence Curve and Its Role in Robust Estimation

Organization or authors: Frank R. Hampel, ETH Zürich
Source type: Original peer-reviewed paper
Publication or effective date: 1974
Version: Journal of the American Statistical Association 69(346), 383–393
URL or DOI: https://doi.org/10.1080/01621459.1974.10482962
Jurisdiction: Statistical methodology; not jurisdiction-specific
Supports: The robust-estimation and bounded-influence motivation associated with Hampel's work.
Limitations: Does not define this package's moving-window API, causal default, threshold, boundary, zero-MAD, or repair policy.

NIST-MAD — Median Absolute Deviation

Organization or authors: National Institute of Standards and Technology
Source type: Official statistical reference documentation
Publication or effective date: Maintained reference; accessed 2026-07-22
Version: Dataplot reference manual
URL or DOI: https://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/mad.htm
Jurisdiction: General statistical guidance
Supports: MAD definition and scaled MAD convention MAD / 0.6745 under a normal reference model.
Limitations: Not a market-data detector specification and does not justify a universal threshold.

NIST-OUTLIERS — Detection of Outliers

Organization or authors: National Institute of Standards and Technology
Source type: Official statistical handbook
Publication or effective date: Maintained reference; accessed 2026-07-22
Version: e-Handbook section 1.3.5.17
URL or DOI: https://www.itl.nist.gov/div898/handbook/eda/section3/eda35h.htm
Jurisdiction: General statistical guidance
Supports: Modified robust scores and the warning that multiple outliers can cause masking.
Limitations: Does not identify whether a financial-market print is erroneous.

PEARSON-2016 — Generalized Hampel Filters

Organization or authors: Ronald K. Pearson, Yrjö Neuvo, Jaakko Astola, Moncef Gabbouj
Source type: Peer-reviewed open-access research article
Publication or effective date: 2016
Version: EURASIP Journal on Advances in Signal Processing 2016:87
URL or DOI: https://doi.org/10.1186/s13634-016-0383-6
Jurisdiction: Signal processing; not market-specific
Supports: Standard Hampel-filter framing as a median-based local outlier filter and the existence of materially different generalized variants.
Limitations: The standard symmetric-window discussion is retrospective; it does not make centered filtering live-safe.

NYSE-TAQ-4.3 — Daily TAQ Client Specifications

Organization or authors: New York Stock Exchange
Source type: Official exchange data-product specification
Publication or effective date: 2026-03-03
Version: 4.3
URL or DOI: https://www.nyse.com/publicdocs/nyse/data/Daily_TAQ_Client_Spec_v4.3.pdf
Jurisdiction: U.S. listed-equity consolidated market data represented in the product
Supports: Daily Trades ordering by symbol, time, and message sequence; trade source and conditions; correction indicator values; sequence number and trade ID fields. These are examples of evidence needed to investigate a suspected bad print.
Limitations: Data are licensed; the specification alone does not confirm any particular historical tick as erroneous.

Evidence and data note

All distributed fixtures in this package are synthetic. No Financial Modeling Prep payload, licensed TAQ record, credential, or provider screenshot is included. A public historical example remains deferred until a raw print and an authoritative linked correction/cancel/error record can be legally reproduced together.

Implementation

Full dependency-light reference implementations in both supported languages.

hampelFilter.ts

export type HampelMode = "causal" | "centered";
export type RepairPolicy = "none" | "median";

export type HampelOptions = {
  windowRadius?: number;
  threshold?: number;
  scale?: number;
  minHistory?: number;
  mode?: HampelMode;
  repair?: RepairPolicy;
};

export type HampelPoint = {
  index: number;
  value: number | null;
  mode: HampelMode;
  windowStart: number;
  windowEnd: number;
  windowCount: number;
  median: number | null;
  mad: number | null;
  scaledMad: number | null;
  score: number | null;
  threshold: number;
  flagged: boolean;
  status: string;
  lookaheadUsed: boolean;
  suggestedReplacement: number | null;
  output: number | null;
};

function median(values: number[]): number {
  const sorted = [...values].sort((a, b) => a - b);
  const middle = Math.floor(sorted.length / 2);
  return sorted.length % 2 === 1
    ? sorted[middle]
    : (sorted[middle - 1] + sorted[middle]) / 2;
}

export function hampelFilter(
  values: Array<number | null>,
  options: HampelOptions = {},
): HampelPoint[] {
  const {
    windowRadius = 3,
    threshold = 3,
    scale = 1.4826,
    minHistory = 3,
    mode = "causal",
    repair = "none",
  } = options;

  if (!Number.isInteger(windowRadius) || windowRadius < 1) {
    throw new Error("windowRadius must be a positive integer");
  }
  if (!Number.isInteger(minHistory) || minHistory < 1) {
    throw new Error("minHistory must be a positive integer");
  }
  if (mode !== "causal" && mode !== "centered") {
    throw new Error("mode must be 'causal' or 'centered'");
  }
  if (repair !== "none" && repair !== "median") {
    throw new Error("repair must be 'none' or 'median'");
  }
  if (!Number.isFinite(threshold) || threshold <= 0) {
    throw new Error("threshold must be finite and positive");
  }
  if (!Number.isFinite(scale) || scale <= 0) {
    throw new Error("scale must be finite and positive");
  }
  if (values.some((value) => value !== null && !Number.isFinite(value))) {
    throw new Error("values must be finite numbers or null");
  }

  return values.map((value, index) => {
    const windowStart = Math.max(0, index - windowRadius);
    const endExclusive = mode === "causal"
      ? index + 1
      : Math.min(values.length, index + windowRadius + 1);
    const window = values
      .slice(windowStart, endExclusive)
      .filter((candidate): candidate is number => candidate !== null);
    const lookaheadUsed = mode === "centered" && endExclusive > index + 1;

    let center: number | null = window.length ? median(window) : null;
    let mad: number | null = center === null
      ? null
      : median(window.map((candidate) => Math.abs(candidate - center!)));
    let scaledMad: number | null = mad === null ? null : scale * mad;
    let score: number | null = null;
    let flagged = false;
    let status = "eligible";

    if (value === null) {
      status = "missing";
      center = null;
      mad = null;
      scaledMad = null;
    } else if (window.length < minHistory) {
      status = "insufficient_history";
    } else {
      const deviation = Math.abs(value - center!);
      if (scaledMad === 0) {
        score = deviation === 0 ? 0 : Number.POSITIVE_INFINITY;
        status = deviation === 0 ? "zero_mad_match" : "zero_mad_deviation";
      } else {
        score = deviation / scaledMad!;
      }
      flagged = score > threshold;
    }

    const suggestedReplacement = flagged ? center : null;
    const output = repair === "median" && flagged ? suggestedReplacement : value;
    return {
      index,
      value,
      mode,
      windowStart,
      windowEnd: endExclusive - 1,
      windowCount: window.length,
      median: center,
      mad,
      scaledMad,
      score,
      threshold,
      flagged,
      status,
      lookaheadUsed,
      suggestedReplacement,
      output,
    };
  });
}

Playground

The embedded lab now expands to its full document height, keeping the article as the only scroll surface.

Full-height labplaygroundOpen full screen