JuliusBrussee/caveman · error · Error

${where} cohort references a task family that is not in this

Error message

${where} cohort references a task family that is not in this report

What it means

Every case's cohort must reference a task family defined in the same report file. The validator builds familiesByID from report.task_families and throws when item.cohort.task_family_id is absent from it (validate-continuous-improvement.mjs:138-139). Reports are deliberately self-contained: a family id that lives only in another fixture, or nowhere, fails here.

Source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:139

      runs += variant.eligible_runs;
      weighted += (motif.operations.length / variant.signature.length) * variant.eligible_runs;
    }
    if (motif.support_run_count !== runs) throw new Error(`${where} support run count ${motif.support_run_count} != ${runs}`);
    const expectedShare = runs > 0 ? weighted / runs : 0;
    if (Math.abs(motif.structural_cost_share - expectedShare) > 1e-6) {
      throw new Error(`${where} structural_cost_share ${motif.structural_cost_share} != ${expectedShare}`);
    }
  }
  motifCount += report.motifs.length;

  // The causal investigation of every case: the cohort's arms, the traces it
  // selected from them, and the backward hard-dependency slice.
  const unitsByID = new Map(report.analysis_units.map((unit) => [unit.id, unit]));
  const familiesByID = new Map(report.task_families.map((family) => [family.id, family]));
  for (const item of report.cases) {
    const where = `report fixture ${reportPaths[index]}: case ${item.id}`;
    const family = familiesByID.get(item.cohort.task_family_id);
    if (!family) throw new Error(`${where} cohort references a task family that is not in this report`);
    const familyUnits = new Set(family.analysis_unit_ids);
    if (item.cohort.family_unit_count !== familyUnits.size) throw new Error(`${where} cohort family unit count disagrees with the task family`);
    const roles = item.cohort.arms.map((arm) => arm.role);
    if (roles.join(",") !== "baseline,alternative") throw new Error(`${where} cohort arms are not baseline then alternative`);
    const armVariants = new Map();
    let comparedUnits = 0;
    for (const arm of item.cohort.arms) {
      const variant = variantsByID.get(arm.variant_id);
      if (!variant) throw new Error(`${where} cohort arm ${arm.role} references a workflow variant that is not in this report`);
      if (variant.task_family_id !== family.id) throw new Error(`${where} cohort arm ${arm.role} uses a variant of another task family`);
      armVariants.set(arm.role, variant);
      comparedUnits += arm.unit_count;
    }
    const excluded = item.cohort.excluded_units.reduce((total, exclusion) => total + exclusion.unit_count, 0);
    if (comparedUnits + excluded !== item.cohort.family_unit_count) {
      throw new Error(`${where} cohort arms (${comparedUnits}) plus exclusions (${excluded}) do not account for its ${item.cohort.family_unit_count} family units`);
    }

View on GitHub (pinned to 766dce6b13)

Solutions

Fix cohort.task_family_id so it byte-matches an id in this report's task_families array.
If the family object was accidentally deleted, restore it to report.task_families (with its analysis_unit_ids).
If the family genuinely belongs to another report, move the case to that report instead of cross-referencing.
Re-run the validator.

Example fix

// before — report defines task family "family-codegen"
"cohort": { "task_family_id": "family-code-gen", ... }
// after
"cohort": { "task_family_id": "family-codegen", ... }

Defensive patterns

Strategy: validation

Validate before calling

const danglingCohortFamilies = (report) => {
  const ids = new Set(report.task_families.map((f) => f.id));
  return report.cases.filter((c) => !ids.has(c.cohort.task_family_id)).map((c) => c.id);
};

Try / catch

try {
  execFileSync(process.execPath, [VALIDATOR, reportPath, spansPath]);
} catch (err) {
  if (/cohort references a task family/.test(err.message)) {
    failCI(`dangling family id — check task_families in the same file: ${err.message}`);
  } else throw err;
}

Prevention

Keep every case and the task family it cites in the same report fixture.
When renaming family ids, grep all fixtures for the old id before committing.
Validate fixtures locally before pushing; CI runs this validator on every build.

When it happens

Trigger: A case block whose cohort.task_family_id is a typo or renamed id (family-code-gen vs family-codegen); the task_families entry was deleted while its cases were kept; the case was copied from a different report fixture that defined the family.

Common situations: Copy-pasting a case between fixture files; renaming family ids in a refactor without sweeping case references; splitting one big fixture into per-topic fixtures and leaving cross-references behind.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/abc72b516d1cf4fa. Report an issue: GitHub.