{"record":{"id":"91d5fab3e7d79be1","repo":"JuliusBrussee/caveman","slug":"where-cohort-arms-are-not-baseline-then-alterna","errorCode":null,"errorMessage":"${where} cohort arms are not baseline then alternative","messagePattern":"(.+?) cohort arms are not baseline then alternative","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/shared/contracts/scripts/validate-continuous-improvement.mjs","lineNumber":143,"sourceCode":"    const expectedShare = runs > 0 ? weighted / runs : 0;\n    if (Math.abs(motif.structural_cost_share - expectedShare) > 1e-6) {\n      throw new Error(`${where} structural_cost_share ${motif.structural_cost_share} != ${expectedShare}`);\n    }\n  }\n  motifCount += report.motifs.length;\n\n  // The causal investigation of every case: the cohort's arms, the traces it\n  // selected from them, and the backward hard-dependency slice.\n  const unitsByID = new Map(report.analysis_units.map((unit) => [unit.id, unit]));\n  const familiesByID = new Map(report.task_families.map((family) => [family.id, family]));\n  for (const item of report.cases) {\n    const where = `report fixture ${reportPaths[index]}: case ${item.id}`;\n    const family = familiesByID.get(item.cohort.task_family_id);\n    if (!family) throw new Error(`${where} cohort references a task family that is not in this report`);\n    const familyUnits = new Set(family.analysis_unit_ids);\n    if (item.cohort.family_unit_count !== familyUnits.size) throw new Error(`${where} cohort family unit count disagrees with the task family`);\n    const roles = item.cohort.arms.map((arm) => arm.role);\n    if (roles.join(\",\") !== \"baseline,alternative\") throw new Error(`${where} cohort arms are not baseline then alternative`);\n    const armVariants = new Map();\n    let comparedUnits = 0;\n    for (const arm of item.cohort.arms) {\n      const variant = variantsByID.get(arm.variant_id);\n      if (!variant) throw new Error(`${where} cohort arm ${arm.role} references a workflow variant that is not in this report`);\n      if (variant.task_family_id !== family.id) throw new Error(`${where} cohort arm ${arm.role} uses a variant of another task family`);\n      armVariants.set(arm.role, variant);\n      comparedUnits += arm.unit_count;\n    }\n    const excluded = item.cohort.excluded_units.reduce((total, exclusion) => total + exclusion.unit_count, 0);\n    if (comparedUnits + excluded !== item.cohort.family_unit_count) {\n      throw new Error(`${where} cohort arms (${comparedUnits}) plus exclusions (${excluded}) do not account for its ${item.cohort.family_unit_count} family units`);\n    }\n\n    const selectedPerArm = new Map();\n    for (const trace of item.representative_traces) {\n      const variant = armVariants.get(trace.arm);\n      if (!variant || variant.id !== trace.variant_id) throw new Error(`${where} representative trace names a variant that is not that arm's variant`);","sourceCodeStart":125,"sourceCodeEnd":161,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/766dce6b1394ebb56a3090748d5a0240a5aefb36/packages/shared/contracts/scripts/validate-continuous-improvement.mjs#L125-L161","documentation":"A case cohort must carry exactly two arms ordered baseline first, alternative second — the comparison direction that every downstream check (causal slice on baseline, opportunity pairing) depends on. The validator joins the arm roles and compares the result against the literal 'baseline,alternative' (validate-continuous-improvement.mjs:142-143), so a swapped pair, a missing arm, an extra arm, or a role typo all fail with this one message.","triggerScenarios":"arms array ordered [alternative, baseline]; a third arm appended; a role spelled 'control'/'treatment'/'base'; a single-arm cohort. The JSON schema does not pin the sequence, so these shapes reach the validator and are rejected here.","commonSituations":"Hand-authoring cohorts with intuitive role names; reordering arms while polishing a fixture; porting a fixture from an older format that allowed more arms.","solutions":["Reorder the arms so the baseline arm is first and the alternative arm is second.","Ensure there are exactly two arms and their role fields are exactly \"baseline\" and \"alternative\".","Re-run the validator — arm-order errors also cascade into error 614, so fix this first."],"exampleFix":"// before\n\"arms\": [\n  { \"role\": \"alternative\", \"variant_id\": \"wf-build-02\", \"unit_count\": 25 },\n  { \"role\": \"baseline\", \"variant_id\": \"wf-build-01\", \"unit_count\": 24 }\n]\n// after\n\"arms\": [\n  { \"role\": \"baseline\", \"variant_id\": \"wf-build-01\", \"unit_count\": 24 },\n  { \"role\": \"alternative\", \"variant_id\": \"wf-build-02\", \"unit_count\": 25 }\n]","handlingStrategy":"validation","validationCode":"const armSequenceOK = (report) =>\n  report.cases.every((c) => c.cohort.arms.map((a) => a.role).join(',') === 'baseline,alternative');","typeGuard":null,"tryCatchPattern":"try {\n  execFileSync(process.execPath, [VALIDATOR, reportPath, spansPath]);\n} catch (err) {\n  if (/not baseline then alternative/.test(err.message)) {\n    failCI(`arm roles/order wrong — exactly [baseline, alternative]: ${err.message}`);\n  } else throw err;\n}","preventionTips":["Treat arm order as part of the fixture's contract, not presentation.","When cloning a case, re-check the arms array before anything else — many later checks depend on it.","Avoid inventing role names; only baseline and alternative are accepted."],"tags":["fixtures","cross-validation","cohort","arms","ordering"],"backgroundTag":"enum-sequence-violation","analyzedSha":"766dce6b1394ebb56a3090748d5a0240a5aefb36","analyzedAt":"2026-08-18T03:14:35.516Z","contentChangedAt":"2026-08-18T03:14:35.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}