{"record":{"id":"80a151b9ca78b00f","repo":"JuliusBrussee/caveman","slug":"where-names-an-opportunity-that-is-not-in-this","errorCode":null,"errorMessage":"${where} names an opportunity that is not in this report","messagePattern":"(.+?) names an opportunity that is not in this report","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/shared/contracts/scripts/validate-continuous-improvement.mjs","lineNumber":229,"sourceCode":"    }\n\n    // A case that proposes nothing must carry nothing: no operator, no eval\n    // pack, no replay. Anything else would present an abstention as a proposal.\n    if (item.status === \"diagnosis_only\") {\n      if (item.change_set.operator_id !== \"\" || item.eval_pack.id !== \"\" || item.replay.status !== \"not_run\") {\n        throw new Error(`${where} is diagnosis-only but carries a change set, eval pack or replay`);\n      }\n      continue;\n    }\n    if (item.change_set.case_id !== item.id || item.eval_pack.case_id !== item.id) {\n      throw new Error(`${where} change set or eval pack is bound to another case`);\n    }\n\n    // The case must investigate the pair its own opportunity named. A cohort\n    // built from a differently-derived pair would describe one comparison and\n    // measure another.\n    const source = report.opportunities.find((opportunity) => opportunity.id === item.opportunity_id);\n    if (!source) throw new Error(`${where} names an opportunity that is not in this report`);\n    if (armVariants.get(\"baseline\").id !== source.current_variant_id || armVariants.get(\"alternative\").id !== source.alternative_variant_id) {\n      throw new Error(`${where} cohort arms (${armVariants.get(\"baseline\").id}/${armVariants.get(\"alternative\").id}) are not the variants opportunity ${source.id} named`);\n    }\n\n    // Every required grader is one the recorded interpreter actually computes.\n    // `json_schema` is deliberately absent: over a typed interpreter result it\n    // could only assert that a hash is non-empty, which is a grader that cannot\n    // fail and therefore inflates how much a replay was checked.\n    const knownGraders = new Set([\"exact_match\", \"tool_order\", \"tool_count\", \"guard_respected\"]);\n    for (const grader of item.eval_pack.graders) {\n      if (!knownGraders.has(grader)) throw new Error(`${where} requires grader ${grader}, which the recorded interpreter does not compute`);\n    }\n\n    // Every dataset case — recorded or generated — must name a real analysis\n    // unit of this case's own task family, and its generator and perturbation\n    // must agree. A generated case whose source unit cannot be resolved would be\n    // invented evidence wearing an adversarial label.\n    const dataset = item.eval_pack.dataset;","sourceCodeStart":211,"sourceCodeEnd":247,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/packages/shared/contracts/scripts/validate-continuous-improvement.mjs#L211-L247","documentation":"Thrown when a case's opportunity_id does not match any opportunity in report.opportunities. Cases are derived from opportunities recorded in the same report; a case citing a foreign or missing opportunity breaks the report's internal derivation chain.","triggerScenarios":"Typo in opportunity_id; deleting an opportunity from the report while its case remains; copying a case between two different reports.","commonSituations":"Splitting a report into two files; renaming opportunity ids; generating cases from a different (older) opportunity list than the one embedded in the report.","solutions":["Check report.opportunities for the id the case should reference and correct opportunity_id.","If the opportunity was intentionally removed, remove the case that cites it too.","If cases and opportunities are generated by different steps, regenerate them together from one source list."],"exampleFix":"// before\nitem.opportunity_id = \"opp-041\"\nreport.opportunities = [{id: \"opp-41\"}, ...]\n\n// after\nitem.opportunity_id = \"opp-41\"","handlingStrategy":"validation","validationCode":"const known = new Set(report.opportunities.map((o) => o.id));\nconst resolvable = known.has(item.opportunity_id);","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Generate cases in the same pass that generates opportunities so ids come from one source.","When deleting an opportunity, cascade-delete its cases; when splitting reports, move both together."],"tags":["validation","referential-integrity","report-consistency"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}