# D08-F04-A03 — Matrix-Profile Motif Discovery

> **Domain:** Geometric Chart Patterns  
> **Family:** Pattern Matching  
> **Status:** Video-ready  
> **Canonical variant:** package-selected, auditable quantitative matching algorithm with explicit time, normalization, threshold, tie, and invalid-state rules

## Executive summary

Turn a long series into a nearest-neighbor distance profile and recover repeated subsequences without counting trivial overlaps.

For every subsequence start, the matrix profile stores the distance to its nearest eligible neighbor and the neighbor's index. The lowest profile value identifies a motif pair under the selected distance and exclusion zone. The package asks one narrow question and returns an auditable diagnostic. It does not forecast price, estimate probability, select a position, or establish profitability.

The topic includes Python and TypeScript references, one persisted expected result, adversarial tests, six static teaching visuals, a Mermaid reasoning flow, and a self-contained guided lab with dense synthetic scenarios.

![Annotated canonical result](visuals/static/article-hero.svg)

## Problem and financial relevance

Visual pattern labels are easy to draw after the full chart is visible. Production research must instead decide what data existed, what transformation was used, when a candidate became knowable, which equality policy applied, and why a near match failed. Matrix-Profile Motif Discovery becomes useful as an engineering object only after those decisions are frozen.

Appropriate uses include chart annotation, research feature generation, quality-controlled scanning, algorithm comparison, and education. Passing the detector or matcher proves only that code implements this package's definition.

## Learning objectives

After completing this topic, a beginner can:

- explain Matrix-Profile Motif Discovery in plain language before using notation;
- reproduce the selected formula and canonical expected result;
- identify normalization, causality, threshold, and tie decisions that alter output;
- implement matching Python and TypeScript behavior;
- distinguish searching, candidate, confirmed, rejected, matched, and ineligible states where applicable;
- use the playground to predict a state change before revealing it;
- explain why a good geometric or distance match is not evidence of future return.

## Prerequisites

Read a numeric sequence, calculate a mean and slope, and distinguish an observation's event index from the later time at which derived evidence can be known. The lab teaches the specialized geometry or distance calculation from the beginning.

## Definition and intuition

For every subsequence start, the matrix profile stores the distance to its nearest eligible neighbor and the neighbor's index. The lowest profile value identifies a motif pair under the selected distance and exclusion zone.

![The components used by the selected contract](visuals/static/algorithm-anatomy.svg)

## Scope and selected convention

The canonical input uses finite synthetic sequences with explicit roles, per-sequence population z-normalization where selected, algorithm-specific constraints, and deterministic tie rules. The exact contract is:

`P_i=min_{j:|i-j|>e} ||z(T_i,m)-z(T_j,m)||_2; I_i=argmin_j; e=ceil(m/2) by package default.`

This is a transparent implementation choice where alternatives exist. Without an exclusion zone, overlapping copies of the same observation path dominate. The motif is nearest under the chosen length and normalization; it is not automatically a chart pattern or tradable event.

### Nearby methods

Template matching searches for a supplied query. Matrix profile discovers nearest subsequence neighbors across the series. Shapelets optimize class discrimination instead of recurrence alone.

![A comparison with nearby methods](visuals/static/method-comparison.svg)

## Data and time contract

The complete schema and invalid-state policy are in [DATA-CONTRACT.md](DATA-CONTRACT.md). The central rule is that transformations may use only information available inside their declared data role. Corrections require replay; missing observations are not silently filled; raw and adjusted price bases are never mixed.

## Mathematical formulation

The selected formula is:

`P_i=min_{j:|i-j|>e} ||z(T_i,m)-z(T_j,m)||_2; I_i=argmin_j; e=ceil(m/2) by package default.`

Every symbol and boundary is connected to the fixture, implementation, tests, and visuals. The calculation uses full precision. A formatted line that looks equal after rounding is not equal unless the unrounded comparison says so.

![Canonical arithmetic and diagnostics](visuals/static/calculation-ledger.svg)

## Algorithm

1. Validate finite sequences, roles, lengths, parameters, and any training/evaluation boundary.
2. Apply the package-selected normalization inside each eligible sequence or window.
3. Enumerate candidate alignments, windows, subsequences, or threshold splits without using future or evaluation labels.
4. Calculate the topic-specific local and aggregate distance or information criterion.
5. Apply the declared exclusion, radius, constant-window, and tie policies.
6. Return the best result plus the full profile, path, matrix, or training diagnostics needed for audit.
7. Preserve raw indexes and unrounded values; round only in the reader.
8. Test scale/offset invariance, boundary equality, malformed data, and deterministic ties.
9. Separate definition fidelity from historical association or trading usefulness.

Implementation-independent pseudocode is in [examples/PSEUDOCODE.md](examples/PSEUDOCODE.md).

## Canonical synthetic worked example

The canonical fixture is repository-authored synthetic teaching data. Its expected result is persisted independently from the runtime facades:

`motif_pair=[20, 72]; motif_distance=0.000000000000; exclusion_zone=8`

The fixture is deliberately shaped to make the governing boundary visible. It is not a selected historical winner and it makes no performance claim.

![Candidate, boundary, and output sequence](visuals/static/causal-timeline.svg)

## Interpretation boundary

Without an exclusion zone, overlapping copies of the same observation path dominate. The motif is nearest under the chosen length and normalization; it is not automatically a chart pattern or tradable event. A detected structure or small distance is conditional on the data field, adjustment basis, window, normalization, pivot rule, thresholds, and search universe. Changing those choices changes the question.

![Acceptance and rejection boundaries](visuals/static/failure-boundary.svg)

## Implementation guide

- [Python reference](implementations/python/algorithm.py)
- [TypeScript reference](implementations/typescript/algorithm.ts)
- [Canonical fixture](examples/worked-example.json)
- [Teaching scenarios](datasets/scenarios.json)
- [Python tests](tests/test_algorithm.py)
- [TypeScript tests](tests/algorithm.test.ts)

The topic facades delegate reviewed arithmetic to the family engine. They remain short so readers can see the selected topic and parameters without duplicating ten divergent implementations.

## Testing and validation

Tests cover the canonical expected result, malformed and nonfinite values, boundary equality, invariance where mathematically valid, causal prefix behavior, deterministic ties, constant-window rejection, radius/exclusion constraints, and Python/TypeScript fixture parity. These tests establish definition fidelity, not empirical usefulness.

## Edge cases and failure modes

- Back-plotted extrema can reveal future confirmation if event and knowledge indexes are collapsed.
- Constant sequences make z-normalization undefined rather than perfectly similar.
- One threshold or window chosen after inspecting outcomes creates selection bias.
- Split-like discontinuities, roll gaps, unit changes, stale prices, and data corrections can create excellent but meaningless geometry.
- Flexible warping, permissive tolerances, and a large search universe increase the chance of an impressive in-sample match.
- A label can remain stable while its reason or evidence becomes stale after correction; caches must retain source and parameter identity.
- Visual similarity, algorithmic match, historical association, and net trading value are four different questions.

## Guided playground

Open the [interactive guided lab](visuals/animated/playground.html). Begin with the complete canonical preview, choose a scenario and parameter setting, predict the next decision, then use Back, Step, Play, Pause, or Reset. The chart, algorithm state, diagnostics, guidance, and audit view update from the same deterministic state.

## Historical-example decision

Decision: **not useful** for this mechanics-first build. A named market period would add provider, security identity, session, adjustment, revision, redistribution, selection, and hindsight questions without improving independent arithmetic. A future historical case should be included only with licensed point-in-time observations, precise parameter vintage, event and availability times, and a predeclared evaluation design.

## Evidence boundary

- Original papers and official documentation support algorithmic primitives and configurable choices.
- The selected thresholds, equality rules, defaults, and fixture are implementation choices or synthetic inputs.
- Displayed expected values are author-derived calculations checked against both languages.
- No empirical prediction, win rate, expected return, or profitability claim is made.

## Related learning

Prerequisite: **Causal Pivot Detection** for confirmed extrema and knowledge time. Compare with **Double Top**, **Trendline Breakout**, and the other topics in **Continuation Structures** and **Pattern Matching**. The family overview states when geometry, query search, motif discovery, or supervised discrimination answers the intended question.

## References

See [REFERENCES.md](REFERENCES.md) for source roles, versions, access dates, and limitations, and [research/CLAIM-LEDGER.md](research/CLAIM-LEDGER.md) for the sourced-versus-selected boundary.