{"record":{"id":"175ad5de37b4ca17","repo":"JuliusBrussee/caveman","slug":"where-cohort-arm-arm-role-references-a-workf","errorCode":null,"errorMessage":"${where} cohort arm ${arm.role} references a workflow variant that is not in this report","messagePattern":"(.+?) cohort arm (.+?) references a workflow variant that is not in this report","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/shared/contracts/scripts/validate-continuous-improvement.mjs","lineNumber":148,"sourceCode":"  motifCount += report.motifs.length;\n\n  // The causal investigation of every case: the cohort's arms, the traces it\n  // selected from them, and the backward hard-dependency slice.\n  const unitsByID = new Map(report.analysis_units.map((unit) => [unit.id, unit]));\n  const familiesByID = new Map(report.task_families.map((family) => [family.id, family]));\n  for (const item of report.cases) {\n    const where = `report fixture ${reportPaths[index]}: case ${item.id}`;\n    const family = familiesByID.get(item.cohort.task_family_id);\n    if (!family) throw new Error(`${where} cohort references a task family that is not in this report`);\n    const familyUnits = new Set(family.analysis_unit_ids);\n    if (item.cohort.family_unit_count !== familyUnits.size) throw new Error(`${where} cohort family unit count disagrees with the task family`);\n    const roles = item.cohort.arms.map((arm) => arm.role);\n    if (roles.join(\",\") !== \"baseline,alternative\") throw new Error(`${where} cohort arms are not baseline then alternative`);\n    const armVariants = new Map();\n    let comparedUnits = 0;\n    for (const arm of item.cohort.arms) {\n      const variant = variantsByID.get(arm.variant_id);\n      if (!variant) throw new Error(`${where} cohort arm ${arm.role} references a workflow variant that is not in this report`);\n      if (variant.task_family_id !== family.id) throw new Error(`${where} cohort arm ${arm.role} uses a variant of another task family`);\n      armVariants.set(arm.role, variant);\n      comparedUnits += arm.unit_count;\n    }\n    const excluded = item.cohort.excluded_units.reduce((total, exclusion) => total + exclusion.unit_count, 0);\n    if (comparedUnits + excluded !== item.cohort.family_unit_count) {\n      throw new Error(`${where} cohort arms (${comparedUnits}) plus exclusions (${excluded}) do not account for its ${item.cohort.family_unit_count} family units`);\n    }\n\n    const selectedPerArm = new Map();\n    for (const trace of item.representative_traces) {\n      const variant = armVariants.get(trace.arm);\n      if (!variant || variant.id !== trace.variant_id) throw new Error(`${where} representative trace names a variant that is not that arm's variant`);\n      if (!unitsByID.has(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} is not an analysis unit of this report`);\n      if (!familyUnits.has(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} is not a member of the cohort's task family`);\n      if (!variant.analysis_unit_ids.includes(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} did not run that arm's workflow variant`);\n      selectedPerArm.set(trace.arm, (selectedPerArm.get(trace.arm) ?? 0) + 1);\n    }","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/766dce6b1394ebb56a3090748d5a0240a5aefb36/packages/shared/contracts/scripts/validate-continuous-improvement.mjs#L130-L166","documentation":"Each cohort arm must name a workflow variant defined in the same report: the validator resolves arm.variant_id against variantsByID (report.workflow_variants) and throws when the lookup misses (validate-continuous-improvement.mjs:146-148). Like the task-family check this enforces report self-containment — the baseline/alternative comparison may only use variants the report itself carries.","triggerScenarios":"arm.variant_id is a typo or stale id after a rename (wf-build-2 vs wf-build-02); the workflow_variants entry was deleted while the case was kept; the variant exists only in a sibling fixture file.","commonSituations":"Renaming variant ids in a refactor without sweeping arm references; trimming unused variants from a fixture that still referenced them; merging fixtures where ids collided and were renumbered.","solutions":["Set arm.variant_id to an id that exists in this report's workflow_variants.","If the variant object was accidentally removed, restore it (including signature, eligible_runs and analysis_unit_ids).","Keep the case and both its arm variants in the same report file, then re-run the validator."],"exampleFix":"// before — report carries variant \"wf-build-01\" and \"wf-build-02\"\n{ \"role\": \"baseline\", \"variant_id\": \"wf-build-2\", \"unit_count\": 24 }\n// after\n{ \"role\": \"baseline\", \"variant_id\": \"wf-build-01\", \"unit_count\": 24 }","handlingStrategy":"validation","validationCode":"const danglingArmVariants = (report) => {\n  const ids = new Set(report.workflow_variants.map((v) => v.id));\n  return report.cases.flatMap((c) =>\n    c.cohort.arms.filter((a) => !ids.has(a.variant_id)).map((a) => `${c.id}:${a.role}`));\n};","typeGuard":null,"tryCatchPattern":"try {\n  execFileSync(process.execPath, [VALIDATOR, reportPath, spansPath]);\n} catch (err) {\n  if (/cohort arm .* workflow variant that is not/.test(err.message)) {\n    failCI(`arm points at a variant outside this report: ${err.message}`);\n  } else throw err;\n}","preventionTips":["Keep each case and both its arm variants in one self-contained report file.","On variant renames, sweep cohort arms, motifs, and causal slices in the same commit.","Run the validator as a pre-commit hook so dangling ids never land on main."],"tags":["fixtures","cross-validation","cohort","arms","referential-integrity"],"backgroundTag":"dangling-id-reference","analyzedSha":"766dce6b1394ebb56a3090748d5a0240a5aefb36","analyzedAt":"2026-08-18T03:14:35.516Z","contentChangedAt":"2026-08-18T03:14:35.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}