JuliusBrussee/caveman · error · Error

${where} cohort arms (${comparedUnits}) plus exclusions (${e

Error message

${where} cohort arms (${comparedUnits}) plus exclusions (${excluded}) do not account for its ${item.cohort.family_unit_count} family units

What it means

The cohort must be a complete partition of its family's units: the sum of both arms' unit_count plus the sum of excluded_units[].unit_count must equal cohort.family_unit_count (validate-continuous-improvement.mjs:151-156). Exclusions are how the cohort accounts for units that ran neither arm's variant; any slack or double counting fails here.

Source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:155

    const where = `report fixture ${reportPaths[index]}: case ${item.id}`;
    const family = familiesByID.get(item.cohort.task_family_id);
    if (!family) throw new Error(`${where} cohort references a task family that is not in this report`);
    const familyUnits = new Set(family.analysis_unit_ids);
    if (item.cohort.family_unit_count !== familyUnits.size) throw new Error(`${where} cohort family unit count disagrees with the task family`);
    const roles = item.cohort.arms.map((arm) => arm.role);
    if (roles.join(",") !== "baseline,alternative") throw new Error(`${where} cohort arms are not baseline then alternative`);
    const armVariants = new Map();
    let comparedUnits = 0;
    for (const arm of item.cohort.arms) {
      const variant = variantsByID.get(arm.variant_id);
      if (!variant) throw new Error(`${where} cohort arm ${arm.role} references a workflow variant that is not in this report`);
      if (variant.task_family_id !== family.id) throw new Error(`${where} cohort arm ${arm.role} uses a variant of another task family`);
      armVariants.set(arm.role, variant);
      comparedUnits += arm.unit_count;
    }
    const excluded = item.cohort.excluded_units.reduce((total, exclusion) => total + exclusion.unit_count, 0);
    if (comparedUnits + excluded !== item.cohort.family_unit_count) {
      throw new Error(`${where} cohort arms (${comparedUnits}) plus exclusions (${excluded}) do not account for its ${item.cohort.family_unit_count} family units`);
    }

    const selectedPerArm = new Map();
    for (const trace of item.representative_traces) {
      const variant = armVariants.get(trace.arm);
      if (!variant || variant.id !== trace.variant_id) throw new Error(`${where} representative trace names a variant that is not that arm's variant`);
      if (!unitsByID.has(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} is not an analysis unit of this report`);
      if (!familyUnits.has(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} is not a member of the cohort's task family`);
      if (!variant.analysis_unit_ids.includes(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} did not run that arm's workflow variant`);
      selectedPerArm.set(trace.arm, (selectedPerArm.get(trace.arm) ?? 0) + 1);
    }
    for (const [arm, count] of selectedPerArm) {
      if (count > 3) throw new Error(`${where} arm ${arm} carries ${count} representative traces, more than three`);
    }

    const slice = item.causal_slice;
    const sliceVariant = variantsByID.get(slice.variant_id);
    if (!sliceVariant) throw new Error(`${where} causal slice references a workflow variant that is not in this report`);

View on GitHub (pinned to 766dce6b13)

Solutions

Verify family_unit_count against the family's unique unit ids first (see error 603), then adjust the arm unit_counts or the excluded_units entries so comparedUnits + excluded equals it.
Make sure no unit is counted twice — an excluded unit must not appear in an arm's unit_count.
Prefer regenerating the fixture from the report generator so the partition is recomputed atomically.

Example fix

// before — arms 24 + 25, exclusions 3, but the family has 52 units
"family_unit_count": 50
// after — 24 + 25 + 3 = 52
"family_unit_count": 52

Defensive patterns

Strategy: validation

Validate before calling

const partitionDrift = (report) =>
  report.cases.filter((c) => {
    const compared = c.cohort.arms.reduce((n, a) => n + a.unit_count, 0);
    const excluded = c.cohort.excluded_units.reduce((n, e) => n + e.unit_count, 0);
    return compared + excluded !== c.cohort.family_unit_count;
  }).map((c) => c.id);

Try / catch

try {
  execFileSync(process.execPath, [VALIDATOR, reportPath, spansPath]);
} catch (err) {
  if (/do not account for/.test(err.message)) {
    failCI(`cohort partition broken — arms + exclusions must equal family units: ${err.message}`);
  } else throw err;
}

Prevention

Fix family_unit_count (error 603) first, then balance arms and exclusions against it.
Never count a unit in both an arm and the exclusion list.
Regenerate the fixture instead of patching one side of the arithmetic by hand.

When it happens

Trigger: An exclusion entry was added after the family grew, without adjusting either side; arm.unit_count is stale after units moved between arms; a unit is counted both in an arm and in excluded_units. Note family_unit_count itself is checked first (error 603), so it agrees with the family's membership list.

Common situations: Editing a fixture to exclude noisy units by hand; generators that compute arm counts and exclusion counts in separate passes; appending new units to the family but only bumping one side of the arithmetic.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/9c3e5b786a4b965e. Report an issue: GitHub.