JuliusBrussee/caveman · error · Error

${where} cohort family unit count disagrees with the task fa

Error message

${where} cohort family unit count disagrees with the task family

What it means

cohort.family_unit_count must equal the number of unique analysis-unit ids on the referenced task family — the validator builds Set(family.analysis_unit_ids) and compares sizes (validate-continuous-improvement.mjs:140-141). The cohort's later population arithmetic (error 607) is anchored on this number, so it must agree with the family's own membership list before anything else is checked.

Source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:141

    }
    if (motif.support_run_count !== runs) throw new Error(`${where} support run count ${motif.support_run_count} != ${runs}`);
    const expectedShare = runs > 0 ? weighted / runs : 0;
    if (Math.abs(motif.structural_cost_share - expectedShare) > 1e-6) {
      throw new Error(`${where} structural_cost_share ${motif.structural_cost_share} != ${expectedShare}`);
    }
  }
  motifCount += report.motifs.length;

  // The causal investigation of every case: the cohort's arms, the traces it
  // selected from them, and the backward hard-dependency slice.
  const unitsByID = new Map(report.analysis_units.map((unit) => [unit.id, unit]));
  const familiesByID = new Map(report.task_families.map((family) => [family.id, family]));
  for (const item of report.cases) {
    const where = `report fixture ${reportPaths[index]}: case ${item.id}`;
    const family = familiesByID.get(item.cohort.task_family_id);
    if (!family) throw new Error(`${where} cohort references a task family that is not in this report`);
    const familyUnits = new Set(family.analysis_unit_ids);
    if (item.cohort.family_unit_count !== familyUnits.size) throw new Error(`${where} cohort family unit count disagrees with the task family`);
    const roles = item.cohort.arms.map((arm) => arm.role);
    if (roles.join(",") !== "baseline,alternative") throw new Error(`${where} cohort arms are not baseline then alternative`);
    const armVariants = new Map();
    let comparedUnits = 0;
    for (const arm of item.cohort.arms) {
      const variant = variantsByID.get(arm.variant_id);
      if (!variant) throw new Error(`${where} cohort arm ${arm.role} references a workflow variant that is not in this report`);
      if (variant.task_family_id !== family.id) throw new Error(`${where} cohort arm ${arm.role} uses a variant of another task family`);
      armVariants.set(arm.role, variant);
      comparedUnits += arm.unit_count;
    }
    const excluded = item.cohort.excluded_units.reduce((total, exclusion) => total + exclusion.unit_count, 0);
    if (comparedUnits + excluded !== item.cohort.family_unit_count) {
      throw new Error(`${where} cohort arms (${comparedUnits}) plus exclusions (${excluded}) do not account for its ${item.cohort.family_unit_count} family units`);
    }

    const selectedPerArm = new Map();
    for (const trace of item.representative_traces) {

View on GitHub (pinned to 766dce6b13)

Solutions

  1. Set family_unit_count to the exact number of unique ids in family.analysis_unit_ids.
  2. Deduplicate family.analysis_unit_ids if it repeats an id.
  3. If you edit one cohort, re-check every other case that references the same family, then re-run the validator.

Example fix

// before — family.analysis_unit_ids holds 14 unique ids
"cohort": { "task_family_id": "family-codegen", "family_unit_count": 12, ... }
// after
"cohort": { "task_family_id": "family-codegen", "family_unit_count": 14, ... }
Defensive patterns

Strategy: validation

Validate before calling

const familyCountDrift = (report) => {
  const byID = new Map(report.task_families.map((f) => [f.id, f]));
  return report.cases.filter((c) =>
    c.cohort.family_unit_count !== new Set(byID.get(c.cohort.task_family_id)?.analysis_unit_ids ?? []).size);
};

Try / catch

try {
  execFileSync(process.execPath, [VALIDATOR, reportPath, spansPath]);
} catch (err) {
  if (/family unit count disagrees/.test(err.message)) {
    failCI(`cohort count out of sync with family membership: ${err.message}`);
  } else throw err;
}

Prevention

When it happens

Trigger: Units were added to or removed from family.analysis_unit_ids without updating cohort.family_unit_count; analysis_unit_ids contains duplicate entries (the Set deduplicates, shrinking the expected count); two cases share one family but only one cohort's count was refreshed.

Common situations: Growing a fixture by appending unit ids to the family but editing counts by hand; fixtures with several cohorts over one family where one went stale; generator writing the family list and the cohort counts in different passes.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/e2ac05eac8063828. Report an issue: GitHub.