Library/Market Breadth and Internals/Concentration and Diffusion

D04-F05-A03 / Complete engineering topic

Effective Number of Constituents

A production-minded guide to Effective Number of Constituents.

D04 · MARKET BREADTH AND INTERNALS
D04-F05-A03Canonical / Tested / Open
D04 / D04-F05
Key concepts

The governed definitions this build depends on. Read them first if a term is unfamiliar.

The operational question

Market breadth is not one idea. A headline move can be driven by a few constituents, a weight vector can be concentrated before any return occurs, and a signal can spread across sectors or factor sleeves without those components being independent. Effective Number of Constituents owns one precise question: how many equally weighted constituents would produce the observed concentration?

A concentration decimal is hard to picture. Effective N translates it into the size of an equally weighted roster with the same concentration. That is a diagnostic and governance outcome, not a forecast, recommendation, legal classification, or promise of diversification.

Intuition before notation

The reciprocal maps concentration back into an intuitive equivalent count: equal weights return the actual count; concentration lowers the effective count. Effective N is a monotone re-expression of HHI, not an independent diversification proof and not the number of names with positive returns.

Where the method sits in Concentration and Diffusion

The family map prevents a category error. Contribution combines beginning weight and realized return. HHI and effective N inspect the weight vector before returns. Diffusion scores direction across a fixed component panel. Choosing the wrong branch may produce valid arithmetic that answers the wrong question.

Freeze the series identity

The selected formula is:

Neff=1i=1Mwi2=1HN_{\mathrm{eff}}=\frac{1}{\sum_{i=1}^{M}w_i^2}=\frac{1}{H}

Take the reciprocal of normalized-weight HHI and report the result beside the actual roster count. The HHI must be positive and computed from one complete normalized nonnegative weight vector. The relevant primary source establishes the underlying convention; the exact diagnostic, translated universe, and parameters remain package choices where stated.

The snapshot needs unique identifiers, finite values, complete coverage, and one point-in-time methodology. as_of is not enough in production: retain observation, availability, and knowledge times plus source owner, revision, and finality. A later corrected roster or classification must not overwrite what was knowable at the earlier cutoff.

Required input

FieldTypeUnit and rule
as_ofISO dateeffective weight snapshot; retain availability and knowledge time separately
security_idstringnonempty and unique under the declared roster policy
weightfinite numbernonnegative fraction; complete vector sums to 1 within 1e-9
readybooleanfalse means the underlying HHI cannot be published

Output and refusal behavior

Return effective_n, actual_n, and the underlying hhi. For a normalized M-constituent vector, 1 <= effective_n <= M.

The reference API validates the weight vector before taking the reciprocal. The lab's equal-weight comparison returns exactly 24, making both the bound and the interpretation visible.

Read the mathematics, not only the score

This is the order-2 effective number associated with squared-share concentration. If 24 unequal weights produce H=0.08980884, then 1/H=11.134761: the roster has the same concentration as roughly 11.1 equal weights.

  • 1 <= N_eff <= M for a complete normalized nonnegative vector.
  • Equal weights produce N_eff = M; a single weight of one produces N_eff = 1.
  • N_eff is a monotone reciprocal of HHI and therefore contains no additional ranking information.
  • Fractional output is expected; it is an equivalent count, not a holding count or independence estimate.

The output must travel with its series identity and diagnostic trace. A bare number strips away the universe, methodology, horizon, tolerance, and evidence status needed to interpret it.

Work the synthetic example

The following is labelled synthetic and all arithmetic is author-derived:

SecurityWeightSquared weight
S010.1782180.03176159
S020.1386140.01921380
S030.1089110.01186158
S040.0891090.00794040
S050.0693070.00480345
S060.0594060.00352907
S070.0495050.00245074
S080.0445540.00198510
… 16 moreincluded in total

The exact expected result is:

JSON
{
  "status": "resolved",
  "effective_n": 11.134761062719665,
  "actual_n": 24,
  "hhi": 0.0898088422703657
}

The canonical distribution and decision boundary

The visual keeps the distribution beside the aggregate. That matters because two identical aggregate values can conceal different component states. The values are synthetic teaching inputs; the displayed result is author-derived arithmetic.

Compare nearby methods

MeasureDefinitionWhat it communicatesMain limitation
Effective N (selected)1/sum(w_i^2)equally weighted count with the same concentrationnot a literal number of independent bets
HHIsum(w_i^2)concentration on a 0–1 scaleless intuitive to non-specialists
Actual roster countnumber of identifiersoperational breadthignores weight imbalance
Entropy effective countexp(-sum(w_i ln w_i))a different sensitivity to small weightsmaterially different diversity order

The comparison is part of the contract. A different numerator, denominator, component universe, weighting rule, tolerance, or signal horizon is not a cosmetic option—it defines a different series.

Implementation walkthrough

The reference algorithm validates before aggregating:

Plain text
validate one complete normalized weight vector
hhi = sum(weight * weight for every constituent)
if hhi <= 0: refuse calculation
effective_n = 1 / hhi
assert 1 <= effective_n <= roster_size
return effective_n, actual roster size, and hhi

Return the actual count, HHI, and effective count together so a fractional equivalent is never mistaken for a literal holding count.

Python and TypeScript consume the same JSON fixture and return the same resolved fields. Invalid or incomplete input raises rather than being repaired. The playground deliberately translates its failure fixture into a visible withheld state because its job is to teach the refusal boundary.

Guided lab

Open the guided playground. Start from the informative canonical state, switch to Comparison, then Failure / boundary. Use Back and Step to inspect how each component enters the result. Compare concentrated and equal-weight rosters and inspect how the reciprocal changes the scale. Reset restores the exact canonical state, and reduced-motion Play advances one deterministic observation without starting a timer.

The calculation flow shows why point-in-time validation precedes aggregation.

Put the diagnostic into an operating workflow

Show actual N and effective N together. The gap is a communication device for weight imbalance, while component-level squared weights remain the audit evidence.

A robust dashboard retains the input snapshot, formula version, parameters, component trace, and refusal reason. It should make methodology or taxonomy changes visible as breaks in series identity rather than quietly splicing them into history.

Edge cases and misuse boundaries

Do not round early, interpret the value as a literal holding count, or compare results from different share-class aggregation policies. Missing components are neither zero nor unchanged. A different roster, taxonomy, factor provider, share-class policy, currency, return basis, horizon, or tolerance defines a different series.

Main takeaway: Effective N translates concentration into an equivalent equal-weight count; it does not estimate independent risk factors.

Passing implementation tests proves that the code matches the selected definition. It does not prove association with future returns, usefulness after costs, causation, or suitability for an investment decision.

Evidence and historical boundary

The source records in REFERENCES.md establish only the claims mapped to them. They do not establish the synthetic numbers or silently define this package's translated universe and parameters.

A named historical case is deferred until the same licensed point-in-time normalized weights required for HHI, plus reproducible reciprocal arithmetic, revision state, and redistribution permission. Deferral is a publication boundary, not a missing-data invitation: the public tutorial uses a transparent synthetic case rather than an irreproducible observed value.

Summary and handoff

You can now calculate N_{\mathrm{eff}}=\frac{1}{\sum_{i=1}^{M}w_i^2}=\frac{1}{H}, identify the series-defining choices, interpret its edge regimes, preserve the audit trace, and reject incomplete snapshots. Continue with Sector Diffusion Index.

Primary sources and exact limitations are recorded in REFERENCES.md.

Calculation Flow — Effective Number of Constituents

Purpose: validate series identity before producing the diagnostic.

Rendering system map…

Takeaway: Missing evidence is a state, never a zero contribution or neutral component.

ReferencesPrimary sources and evidence notes

Expand the source trail, evidence role, and limitations behind the engineering choices.

Claim-to-source map

Material claimEvidenceClassification
HHI squares and sums normalized sharesDOJ-HHIsourced mathematical definition
Reciprocal squared-share concentration has an effective-number interpretationHILL-1973sourced mathematical interpretation; package-translated universe
Index weights depend on the declared index methodologySPDJI-MATHsourced provider convention
Canonical valuesdatasets/canonical-fixture.jsonsynthetic input and author-derived calculation

Every external source ID mapped above is expanded below. Provider or institutional conventions are not presented as universal laws; package translations and parameters remain labelled as implementation choices.

DOJ-HHI — Herfindahl-Hirschman Index

  • Organization or authors: U.S. Department of Justice, Antitrust Division
  • Source type: Official government methodology explanation
  • Publication or effective date: 2023
  • Version: 2023 Merger Guidelines reference
  • URL or DOI: https://www.justice.gov/atr/herfindahl-hirschman-index
  • Accessed: 2026-07-26
  • Jurisdiction: Global methodology unless the limitation states otherwise
  • Supports: HHI as the sum of squared shares and the 0-10,000 percentage-share scale.
  • Limitations: Antitrust thresholds describe market concentration, not investment-index risk labels; this package transfers only the mathematics.

HILL-1973 — Diversity and Evenness: A Unifying Notation and Its Consequences

  • Organization or authors: Mark O. Hill
  • Source type: Original peer-reviewed paper
  • Publication or effective date: 1973
  • Version: Ecology 54(2), DOI 10.2307/1934352
  • URL or DOI: https://esajournals.onlinelibrary.wiley.com/doi/10.2307/1934352
  • Accessed: 2026-07-26
  • Jurisdiction: Global methodology unless the limitation states otherwise
  • Supports: Effective-number interpretation of diversity measures, including the reciprocal of squared-share concentration.
  • Limitations: The paper concerns ecological abundance. Applying the same normalized-share mathematics to index constituents is an explicit package translation.

SPDJI-MATH — S&P Dow Jones Indices: Index Mathematics Methodology

  • Organization or authors: S&P Dow Jones Indices
  • Source type: Official index methodology
  • Publication or effective date: 2026
  • Version: Current methodology retrieved 2026-07-26
  • URL or DOI: https://www.spglobal.com/spdji/en/documents/methodologies/methodology-index-math.pdf
  • Accessed: 2026-07-26
  • Jurisdiction: Global methodology unless the limitation states otherwise
  • Supports: Index weights, weighted component returns, index calculation, capping, and the importance of methodology-specific weights.
  • Limitations: Defines S&P DJI conventions; it does not define this package's Top-N diagnostic, HHI, or diffusion thresholds.
effective_number_of_constituents.ts
type Row = Record<string, unknown>;
const numberValue = (value: unknown, name: string): number => {
  if (typeof value !== "number" || !Number.isFinite(value)) throw new Error(`${name} must be a finite number`);
  return value;
};
const validateRows = (rows: Row[], weighted: boolean): void => {
  if (!rows.length) throw new Error("rows must not be empty");
  const seen = new Set<string>();
  for (const row of rows) {
    const key = row[weighted ? "security_id" : "component_id"];
    if (typeof key !== "string" || !key || seen.has(key)) throw new Error("component identifiers must be nonempty and unique");
    seen.add(key);
    if (row.ready === false) throw new Error("incomplete_component");
  }
};

export function calculate(rows: Row[]) {
  validateRows(rows, true);
  const weights = rows.map(row => numberValue(row.weight, "weight"));
  if (weights.some(w => w < 0) || Math.abs(weights.reduce((a,b)=>a+b,0)-1) > 1e-9) throw new Error("weights must be nonnegative and sum to one");
  const hhi = weights.reduce((sum,w)=>sum+w*w,0);
  return {status:"resolved", effective_n:1/hhi, actual_n:rows.length, hhi};
}
Full-height labplaygroundOpen full screen