By Islam BarakaFor analysts and developers who know decimal interval returns, regular sampling grids, and variance versus volatility. The goal is to reproduce the mechanism, inspect its failure states, and decide what the output can legitimately tell you—not to fit or endorse a trading strategy.
Alternating quote noise can inflate a sum of squared returns. It also creates a pattern in neighboring returns: positive and negative moves tend to offset. A kernel estimator uses lagged return products to include some of that serial information.
This tutorial implements a deliberately precise teaching variant: a finite, zero-extended Bartlett quadratic form. It is not presented as the complete optimally tuned, endpoint-adjusted realized-kernel procedure from the research literature. That boundary matters because “realized kernel” names a family of constructions, not one interchangeable formula.
Open this figure at full size.
Build the lag sums first
For n returns in the window, define γ_h=Σ_(i=h+1)^n r_i r_(i−h). Lag zero is Σr_i², ordinary realized variance. For a chosen integer bandwidth H with 0≤H<n, use
The weights taper linearly. The factor two accounts for the matching positive and negative lag terms in the symmetric quadratic form. There is no division by the number of pairs at each lag.
The highfrequency kernel reference documents the broader lag-weighted family and cites the realized-kernel research. Its bandwidth indexing and optional adjustments differ from this frozen Bartlett convention. Matching names is insufficient when comparing software outputs; match weights, boundaries and normalizations.
Compute a bandwidth-one example
Use returns [0.01,−0.02,0.03,−0.01]. Lag zero is 0.0015. Lag one is (−0.02×0.01)+(0.03×−0.02)+(−0.01×0.03)=−0.0011.
With H=1, the weight is 1/2, so the doubled correction is 2×(1/2)×−0.0011=−0.0011. The estimate is 0.0004, compared with RV=0.0015. At H=0, there are no lag corrections and RK equals RV exactly.
The smaller number is not automatically the “true volatility.” It shows how negative adjacent products change this estimator. Positive lag products can move it in the other direction. Neither direction proves that a particular component of the observed path was noise.
Open this figure at full size.
Download the exact worked input and expected values.
Why this particular construction is nonnegative
Extend returns by zero outside the window. Form every overlapping sum of H+1 adjacent returns, including boundary sums. Then
Expanding the squares counts each lag-zero term H+1 times and each lag-h pair H+1−h times. Dividing by H+1 gives exactly the Bartlett weights above. This author-derived identity proves nonnegativity for the declared finite construction.
That proof should not be generalized to every possible kernel or arbitrary lag weighting. The runtime checks substantive negativity as a numerical failure and only tolerates negligible roundoff relative to the variance scale. It does not use an absolute value to conceal an invalid estimate.
Bandwidth is not the observation window
The window determines which returns belong to the measurement period. H determines how far apart two returns may be while contributing a lag product. Increasing H must not silently shorten or lengthen the underlying period.
| Control or boundary | What changes |
|---|---|
| H=0 | Exactly realized variance |
| H=1 | Only adjacent-return correction |
| Larger H | Additional lag products and changed taper weights |
| Longer observation window | More returns and products at each retained lag |
| H≥n or fractional H | Rejected by this contract |
Open this figure at full size.
A useful experiment with the playground
Keep the 64-return synthetic path fixed and vary H. Inspect the lag table: each row shows γ_h, its weight and its doubled weighted contribution. The displayed output must equal the sum of that table, and the chart must change when the calculated sequence changes.
Now compare a path with alternating measurement disturbance to the clean control. Ask which negative lag products offset the extra squared-return contribution. This is a controlled observation-noise experiment, not a claim that all negative autocorrelation in markets is microstructure noise.
Step adds one observed return while preserving the chosen H and window. Set the bandwidth control to H=0 for a direct RV reference check; the boundary scenario separately uses a zero-return path. Invalid bandwidths produce an explanatory error rather than being silently capped.
Open this figure at full size.
Open the standalone guided playground. The embedded playground and runnable code are available on this page. Download the 64-observation teaching input.
What production use would still require
Realized-kernel research addresses bandwidth selection, noise assumptions, endpoint treatment and asymptotic behavior. This finite lab does not fit a noise model, estimate an optimal bandwidth, jitter endpoints or promise consistency under every observation regime. Its purpose is to make the lag-correction mechanism reproducible.
Use finite decimal returns on a declared regular grid. If timestamps are supplied, spacing and ordering are checked. Missing observations need an explicit upstream policy. The optional scale A defaults to 1 and multiplies every lag contribution consistently; annualization must correspond to the actual measured period.
The direct implementation costs O(nwH) for n rolling observations, window w and bandwidth H. Optimizations can reuse lag products, but boundary bookkeeping deserves tests. Independent checks compare the output with the full Toeplitz quadratic form, not just another copy of the same lag loop.
Start with realized variance, then use this construction to understand what serial corrections buy and what assumptions they add. The real value is being able to point to a lag, a weight and a contribution when an estimate changes.
Reproduce and inspect the calculation
The Python and TypeScript tabs contain standalone implementations, not imports into an unseen runtime. Both expose calculate(input_data). Feed the worked JSON's input object into that entry point. For the longer experiment, use the teaching-path JSON directly.
import json
from pathlib import Path
from realized_kernel import calculate
data = json.loads(Path("teaching-path.json").read_text())
result = calculate(data)
print(result["latest"])
import {calculate} from './realized_kernel.ts';
const result = calculate(inputData); // inputData is the downloaded JSON object
console.log(result.latest);
Place the downloaded input beside your script and the standalone source on its import path. The Python reference uses the standard library; the TypeScript reference has no external runtime dependency. Shared tests include independent numeric anchors, valid boundaries, rejected inputs and cross-language output comparisons. They establish arithmetic, not forecasting performance.
Evidence and scope
This article uses authored synthetic calculations and primary technical references, reviewed 2026-09-10. Historical market examples are deferred until identity, adjustment basis, chronology and redistribution rights can be verified. No personal trading history or search-ranking superiority is asserted.
Finite zero-extended Bartlett quadratic form; gamma_h=sum_(i=h+1)^w r_i r_(i-h). Not the full optimally tuned/jittered realized-kernel estimator. Integer 0<=H<w.
Continue the investigation
- Realized Variance: compare its assumptions and information boundary before comparing the numbers.
- Jump-Variation Detector: compare its assumptions and information boundary before comparing the numbers.
Rendered from the canonical Mermaid sources linked by this article.
Realized Kernel — calculation-flow
Realized Kernel — decision-boundary
ReferencesPrimary sources and evidence notesExpand the source trail, evidence role, and limitations behind the engineering choices.
Expand the source trail, evidence role, and limitations behind the engineering choices.
Reviewed 2026-09-10. Primary technical documentation and papers; synthetic arithmetic is author-derived. This is a targeted source review, not a verified review of Google's top ten results and not a claim of ranking superiority.
- S1: highfrequency — rKernelCov estimator documentation — accessed 2026-09-10. Rolling official documentation snapshot; exact package versions used for numerical comparisons are recorded in the repair numeric-evidence.json. Supports the definition and declared convention, not investment performance. Jurisdiction: not applicable to this mathematical reference.
Scope of evidence
Finite zero-extended Bartlett quadratic form; gamma_h=sum_(i=h+1)^w r_i r_(i-h). Not the full optimally tuned/jittered realized-kernel estimator. Integer 0<=H<w.
Historical case: deferred. No public provider dataset, historical performance claim, or personal trading anecdote is used. Synthetic examples demonstrate arithmetic, not market efficacy. Sources are not copied as article prose.
Accessed: 2026-09-10.
Supports: estimator definition and the explicitly declared variants.
Limitations: technical documentation does not verify a real market feed, author experience, forecast efficacy or search-result superiority. Original-paper access limitations are recorded in the repair report.
Full dependency-light reference implementations in both supported languages.
/** Standalone D10-F02-A05 reference. Generated from validated D10 v2 source. */
export class ContractError extends Error {}
type RecordValue=Record<string, any>;
type Matrix=number[][];
const sum=(x:number[]):number=>x.reduce((a,b)=>a+b,0);
function requireValue(ok: unknown, code: string, message: string): asserts ok {
if (!ok) throw new ContractError(`${code}: ${message}`);
}
function finite(x: unknown, name: string): number {
requireValue(typeof x === 'number' && Number.isFinite(x), 'NUMBER', `${name} must be a finite number`);
return x;
}
function integer(x: unknown, name: string, minimum = 0, maximum = 10000): number {
const v = finite(x, name);
requireValue(Number.isInteger(v) && v >= minimum && v <= maximum, 'INTEGER', `${name} must be an integer in [${minimum}, ${maximum}]`);
return v;
}
function param(p: RecordValue, key: string, fallback: number): number {
return finite(Object.hasOwn(p, key) ? p[key] : fallback, key);
}
function option(p: RecordValue, key: string, fallback: any): any {
return Object.hasOwn(p, key) ? p[key] : fallback;
}
function positive(p: RecordValue, key: string, fallback: number): number {
const v = param(p, key, fallback);
requireValue(v > 0, 'RANGE', `${key} must be positive`);
return v;
}
function vector(value: unknown, name: string, minimum = 1): number[] {
requireValue(Array.isArray(value) && value.length >= minimum, 'SHAPE', `${name} needs ${minimum} or more values`);
return value.map((v, i) => finite(v, `${name}[${i}]`));
}
function timestamps(values: unknown, n: number, name: string, regular = false): number[] {
requireValue(Array.isArray(values) && values.length === n, 'TIME', `${name} must match observation count`);
const result: number[] = [];
for (const value of values) {
requireValue(typeof value === 'string' && /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d{3})?Z$/.test(value), 'TIME', 'use UTC ISO timestamps');
const instant = Date.parse(value);
const canonical = value.length === 20 ? value.replace('Z', '.000Z') : value;
requireValue(Number.isFinite(instant) && Number(value.slice(0, 4)) >= 1000 && new Date(instant).toISOString() === canonical, 'TIME', 'invalid calendar timestamp');
requireValue(!result.length || instant > result[result.length - 1], 'TIME', 'timestamps must strictly increase');
result.push(instant);
}
if (regular && n > 2) requireValue(result.slice(2).every((v, i) => v - result[i + 1] === result[1] - result[0]), 'ALIGNMENT', 'returns need a regular grid');
return result;
}
function seriesResult(series: (RecordValue | null)[], diagnostics: RecordValue): RecordValue {
const at = series.findIndex(v => v !== null);
return {series, latest: series.at(-1) ?? null, ready: series.length > 0 && series.at(-1) !== null,
ready_at: at < 0 ? null : at, diagnostics};
}
function normalQuantile(probability: number): number {
const a = [-39.69683028665376, 220.9460984245205, -275.9285104469687, 138.3577518672690, -30.66479806614716, 2.506628277459239];
const b = [-54.47609879822406, 161.5858368580409, -155.6989798598866, 66.80131188771972, -13.28068155288572];
const c = [-.007784894002430293, -.3223964580411365, -2.400758277161838, -2.549732539343734, 4.374664141464968, 2.938163982698783];
const d = [.007784695709041462, .3224671290700398, 2.445134137142996, 3.754408661907416];
const poly = (co: number[], x: number) => co.slice(1).reduce((v, coefficient) => v * x + coefficient, co[0]);
if (probability < .02425 || probability > .97575) {
const q = Math.sqrt(-2 * Math.log(Math.min(probability, 1 - probability)));
const value = poly(c, q) / (poly(d, q) * q + 1);
return probability < .5 ? value : -value;
}
const q = probability - .5;
return poly(a, q * q) * q / (poly(b, q * q) * q * q + 1);
}
function realized(data: RecordValue, p: RecordValue, kind: string): RecordValue {
const key = kind === 'realized_covariance' ? 'returns_x' : 'returns', r = vector(data[key], key);
const w = integer(option(p, 'window', 24), 'window', kind === 'jump_detector' ? 3 : 2), scale = positive(p, 'annualization_factor', 1);
let y: number[] = [];
if (kind === 'realized_covariance') {
y = vector(data.returns_y, 'returns_y');
const tx = timestamps(data.timestamps_x, r.length, 'timestamps_x', true), ty = timestamps(data.timestamps_y, y.length, 'timestamps_y', true);
requireValue(tx.length === ty.length && tx.every((v, i) => v === ty[i]), 'ALIGNMENT', 'x/y intervals must match exactly');
} else if (Object.hasOwn(data, 'timestamps')) timestamps(data.timestamps, r.length, 'timestamps', true);
const alpha = kind === 'jump_detector' ? param(p, 'alpha', .05) : null;
if (alpha !== null) requireValue(alpha >= 1e-6 && alpha < .5, 'RANGE', 'alpha must be in [0.000001, 0.5)');
const bandwidth = kind === 'realized_kernel' ? integer(option(p, 'bandwidth', Math.min(4, w - 1)), 'bandwidth', 0, w - 1) : 0;
const series = r.map((_, i) => {
if (i + 1 < w) return null;
const x = r.slice(i - w + 1, i + 1), squares = x.map(v => v * v), rv = scale * sum(squares);
const products = x.slice(1).map((v, j) => Math.abs(v * x[j])), bv = scale * Math.PI / 2 * sum(products);
let item: RecordValue;
if (kind === 'realized_variance') item = {variance: rv, volatility: Math.sqrt(rv), contributions: squares.map(v => scale * v)};
else if (kind === 'realized_covariance') {
const paired = x.map((v, j) => scale * v * y[i - w + 1 + j]);
item = {covariance: sum(paired), contributions: paired};
} else if (kind === 'bipower_variation') item = {realized_variance: rv, bipower_variation: bv, signed_difference: rv - bv, jump_variation: Math.max(rv - bv, 0), contributions: products.map(v => scale * Math.PI / 2 * v)};
else if (kind === 'jump_detector') {
const mu43 = .8308609250295592;
const tq = scale ** 2 * w * w / (w - 2) * sum(x.slice(2).map((v, j) => Math.abs(v * x[j + 1] * x[j]) ** (4 / 3))) / mu43 ** 3;
const se = Math.sqrt((Math.PI ** 2 / 4 + Math.PI - 5) * tq / w), z = se > 0 ? (rv - bv) / se : null;
const q = normalQuantile(1 - alpha!);
item = {realized_variance: rv, bipower_variation: bv, tripower_quarticity: tq, standard_error: se, statistic: z,
critical_value: q, jump_detected: z === null ? null : z > q, decision_status: z === null ? 'withheld-zero-quarticity' : 'asymptotic', signed_difference: rv - bv};
} else {
const lags = Array.from({length: bandwidth + 1}, (_, h) => {
const gamma = sum(x.slice(h).map((v, j) => v * x[j])), weight = 1 - h / (bandwidth + 1);
return {lag: h, weight, gamma, contribution: scale * (h === 0 ? 1 : 2) * weight * gamma};
});
const value = sum(lags.map(v => v.contribution));
requireValue(value >= -1e-12 * Math.max(rv, 1e-300), 'NUMERIC', 'Bartlett quadratic form became negative');
item = {realized_kernel: Math.max(value, 0), volatility: Math.sqrt(Math.max(value, 0)), bandwidth, lags};
}
return {...item, window_start: i - w + 1, window_end: i};
});
return seriesResult(series, {causal: true, input_count: r.length, annualization_factor: scale});
}
export function calculate(data: RecordValue): RecordValue {
requireValue(data && typeof data === 'object' && !Array.isArray(data),'SHAPE','input must be an object');
const p=Object.hasOwn(data,'parameters')?data.parameters:{};
requireValue(p && typeof p === 'object' && !Array.isArray(p),'SHAPE','parameters must be an object');
const result=realized(data,p,"realized_kernel");
function check(v:any):void {
if(typeof v==='number')requireValue(Number.isFinite(v),'NUMERIC','nonfinite computed output');
else if(Array.isArray(v))v.forEach(check);
else if(v && typeof v==='object')Object.values(v).forEach(check);
}
check(result);
return {topic_id:"D10-F02-A05",title:"Realized Kernel",parameters:p,...result};
}
The embedded lab now expands to its full document height, keeping the article as the only scroll surface.