{"record":{"id":"e2ac05eac8063828","repo":"JuliusBrussee/caveman","slug":"where-cohort-family-unit-count-disagrees-with-t","errorCode":null,"errorMessage":"${where} cohort family unit count disagrees with the task family","messagePattern":"(.+?) cohort family unit count disagrees with the task family","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/shared/contracts/scripts/validate-continuous-improvement.mjs","lineNumber":141,"sourceCode":"    }\n    if (motif.support_run_count !== runs) throw new Error(`${where} support run count ${motif.support_run_count} != ${runs}`);\n    const expectedShare = runs > 0 ? weighted / runs : 0;\n    if (Math.abs(motif.structural_cost_share - expectedShare) > 1e-6) {\n      throw new Error(`${where} structural_cost_share ${motif.structural_cost_share} != ${expectedShare}`);\n    }\n  }\n  motifCount += report.motifs.length;\n\n  // The causal investigation of every case: the cohort's arms, the traces it\n  // selected from them, and the backward hard-dependency slice.\n  const unitsByID = new Map(report.analysis_units.map((unit) => [unit.id, unit]));\n  const familiesByID = new Map(report.task_families.map((family) => [family.id, family]));\n  for (const item of report.cases) {\n    const where = `report fixture ${reportPaths[index]}: case ${item.id}`;\n    const family = familiesByID.get(item.cohort.task_family_id);\n    if (!family) throw new Error(`${where} cohort references a task family that is not in this report`);\n    const familyUnits = new Set(family.analysis_unit_ids);\n    if (item.cohort.family_unit_count !== familyUnits.size) throw new Error(`${where} cohort family unit count disagrees with the task family`);\n    const roles = item.cohort.arms.map((arm) => arm.role);\n    if (roles.join(\",\") !== \"baseline,alternative\") throw new Error(`${where} cohort arms are not baseline then alternative`);\n    const armVariants = new Map();\n    let comparedUnits = 0;\n    for (const arm of item.cohort.arms) {\n      const variant = variantsByID.get(arm.variant_id);\n      if (!variant) throw new Error(`${where} cohort arm ${arm.role} references a workflow variant that is not in this report`);\n      if (variant.task_family_id !== family.id) throw new Error(`${where} cohort arm ${arm.role} uses a variant of another task family`);\n      armVariants.set(arm.role, variant);\n      comparedUnits += arm.unit_count;\n    }\n    const excluded = item.cohort.excluded_units.reduce((total, exclusion) => total + exclusion.unit_count, 0);\n    if (comparedUnits + excluded !== item.cohort.family_unit_count) {\n      throw new Error(`${where} cohort arms (${comparedUnits}) plus exclusions (${excluded}) do not account for its ${item.cohort.family_unit_count} family units`);\n    }\n\n    const selectedPerArm = new Map();\n    for (const trace of item.representative_traces) {","sourceCodeStart":123,"sourceCodeEnd":159,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/766dce6b1394ebb56a3090748d5a0240a5aefb36/packages/shared/contracts/scripts/validate-continuous-improvement.mjs#L123-L159","documentation":"cohort.family_unit_count must equal the number of unique analysis-unit ids on the referenced task family — the validator builds Set(family.analysis_unit_ids) and compares sizes (validate-continuous-improvement.mjs:140-141). The cohort's later population arithmetic (error 607) is anchored on this number, so it must agree with the family's own membership list before anything else is checked.","triggerScenarios":"Units were added to or removed from family.analysis_unit_ids without updating cohort.family_unit_count; analysis_unit_ids contains duplicate entries (the Set deduplicates, shrinking the expected count); two cases share one family but only one cohort's count was refreshed.","commonSituations":"Growing a fixture by appending unit ids to the family but editing counts by hand; fixtures with several cohorts over one family where one went stale; generator writing the family list and the cohort counts in different passes.","solutions":["Set family_unit_count to the exact number of unique ids in family.analysis_unit_ids.","Deduplicate family.analysis_unit_ids if it repeats an id.","If you edit one cohort, re-check every other case that references the same family, then re-run the validator."],"exampleFix":"// before — family.analysis_unit_ids holds 14 unique ids\n\"cohort\": { \"task_family_id\": \"family-codegen\", \"family_unit_count\": 12, ... }\n// after\n\"cohort\": { \"task_family_id\": \"family-codegen\", \"family_unit_count\": 14, ... }","handlingStrategy":"validation","validationCode":"const familyCountDrift = (report) => {\n  const byID = new Map(report.task_families.map((f) => [f.id, f]));\n  return report.cases.filter((c) =>\n    c.cohort.family_unit_count !== new Set(byID.get(c.cohort.task_family_id)?.analysis_unit_ids ?? []).size);\n};","typeGuard":null,"tryCatchPattern":"try {\n  execFileSync(process.execPath, [VALIDATOR, reportPath, spansPath]);\n} catch (err) {\n  if (/family unit count disagrees/.test(err.message)) {\n    failCI(`cohort count out of sync with family membership: ${err.message}`);\n  } else throw err;\n}","preventionTips":["Edit family.analysis_unit_ids and cohort.family_unit_count in the same change.","Deduplicate analysis_unit_ids — the validator counts unique ids only.","When several cohorts share a family, re-check them all after touching the family."],"tags":["fixtures","cross-validation","cohort","task-family","arithmetic"],"backgroundTag":"derived-field-mismatch","analyzedSha":"766dce6b1394ebb56a3090748d5a0240a5aefb36","analyzedAt":"2026-08-18T03:14:35.516Z","contentChangedAt":"2026-08-18T03:14:35.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}