{"record":{"id":"869edb57dfee55b0","repo":"JuliusBrussee/caveman","slug":"where-is-a-safety-finding-carrying-another-workflow-s","errorCode":null,"errorMessage":"${where} is a safety finding carrying another workflow's metrics","messagePattern":"(.+?) is a safety finding carrying another workflow's metrics","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/shared/contracts/scripts/validate-continuous-improvement.mjs","lineNumber":71,"sourceCode":"  // same money be counted twice under two detectors.\n  const variantIds = new Set(report.workflow_variants.map((variant) => variant.id));\n  for (const opportunity of report.opportunities) {\n    const where = `report fixture ${reportPaths[index]}: opportunity ${opportunity.id}`;\n    for (const key of [\"current_variant_id\", \"alternative_variant_id\"]) {\n      if (opportunity[key] && !variantIds.has(opportunity[key])) {\n        throw new Error(`${where} ${key} ${opportunity[key]} is not a workflow variant of this report`);\n      }\n    }\n    if (opportunity.detector_id === \"dominated-workflow\") {\n      if (!opportunity.current_variant_id || !opportunity.alternative_variant_id) {\n        throw new Error(`${where} compares two workflows without naming both variants`);\n      }\n    }\n    if (opportunity.type === \"safety\") {\n      if (opportunity.alternative_variant_id) throw new Error(`${where} is a safety finding with an alternative variant`);\n      if (opportunity.expected_value !== 0) throw new Error(`${where} is a safety finding carrying expected value ${opportunity.expected_value}`);\n      if (opportunity.alternative_metrics.cost_per_outcome_usd !== null || opportunity.alternative_metrics.eligible_runs !== 0) {\n        throw new Error(`${where} is a safety finding carrying another workflow's metrics`);\n      }\n    }\n  }\n\n  // A relationship is a count, so it must be recomputable from the counts it\n  // carries. Anything a reader cannot re-derive is a claim, not evidence.\n  const themeIds = new Set(report.themes.map((theme) => theme.id));\n  for (const relationship of report.relationships) {\n    const where = `report fixture ${reportPaths[index]}: relationship ${relationship.id}`;\n    if (!themeIds.has(relationship.theme_a_id) || !themeIds.has(relationship.theme_b_id)) {\n      throw new Error(`${where} references a theme that is not in this report`);\n    }\n    if (!(relationship.theme_a_id < relationship.theme_b_id)) {\n      throw new Error(`${where} theme ids are not in canonical order`);\n    }\n    if (relationship.shared_unit_count > relationship.theme_a_unit_count || relationship.shared_unit_count > relationship.theme_b_unit_count) {\n      throw new Error(`${where} shares more units than either theme has`);\n    }","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/packages/shared/contracts/scripts/validate-continuous-improvement.mjs#L53-L89","documentation":"A safety finding must not carry another workflow's alternative metrics: when type === \"safety\", the validator requires alternative_metrics.cost_per_outcome_usd === null and alternative_metrics.eligible_runs === 0, and throws if either is populated. Borrowed cost-per-outcome figures would double-count money already claimed by an efficiency detector, so hazard reports must leave the alternative-metrics block empty.","triggerScenarios":"Running the validator on a report where a type === \"safety\" opportunity has alternative_metrics.cost_per_outcome_usd set to a number or alternative_metrics.eligible_runs set to a non-zero count.","commonSituations":"Generator code default-initializing alternative_metrics with real numbers for every opportunity; a detector converted from efficiency to safety while reusing its metric block; copy-pasted fixtures between opportunity types.","solutions":["Set alternative_metrics.cost_per_outcome_usd to null and alternative_metrics.eligible_runs to 0 on the safety opportunity.","Fix the report generator/detector so alternative_metrics is zeroed for safety findings.","If the metrics are real, re-emit the finding as an efficiency opportunity rather than safety.","Re-run the validator to confirm safety findings carry no borrowed metrics."],"exampleFix":"// before\n{ \"id\": \"opp-3\", \"type\": \"safety\", \"alternative_metrics\": { \"cost_per_outcome_usd\": 0.42, \"eligible_runs\": 17 } }\n// after\n{ \"id\": \"opp-3\", \"type\": \"safety\", \"alternative_metrics\": { \"cost_per_outcome_usd\": null, \"eligible_runs\": 0 } }","handlingStrategy":"validation","validationCode":"if (report.opportunities.some((o) => o.type === \"safety\" && (o.alternative_metrics.cost_per_outcome_usd !== null || o.alternative_metrics.eligible_runs !== 0))) {\n  throw new Error(\"safety finding carries alternative metrics\");\n}","typeGuard":"const hasEmptyAlternativeMetrics = (o) => o.type !== \"safety\" || (o.alternative_metrics.cost_per_outcome_usd === null && o.alternative_metrics.eligible_runs === 0);","tryCatchPattern":"try {\n  await runValidator([reportPath, spansPath]);\n} catch (err) {\n  if (String(err.message).includes(\"carrying another workflow's metrics\")) {\n    console.error(\"Zero/null alternative_metrics on safety findings to avoid double-counted savings.\");\n  }\n  throw err;\n}","preventionTips":["Provide a zeroMetrics() factory used by every safety-opportunity constructor.","Keep alternative_metrics population in one efficiency-only code path.","Diff generated fixtures against expected metric blocks in snapshot tests."],"tags":["validation","fixtures","opportunities","metrics"],"backgroundTag":"invalid-argument-value","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}