{"record":{"id":"6cb90ba3ac1aada2","repo":"JuliusBrussee/caveman","slug":"at-generator-entry-generator-disagrees-with","errorCode":null,"errorMessage":"${at} generator ${entry.generator} disagrees with perturbation ${entry.perturbation}","messagePattern":"(.+?) generator (.+?) disagrees with perturbation (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/shared/contracts/scripts/validate-continuous-improvement.mjs","lineNumber":264,"sourceCode":"    // invented evidence wearing an adversarial label.\n    const dataset = item.eval_pack.dataset;\n    const generatorPerturbation = new Map([\n      [\"recorded_case.v1\", \"none\"],\n      [\"boundary_guard_threshold.v1\", \"guard_threshold_boundary\"],\n      [\"adversarial_misleading_tool_output.v1\", \"misleading_tool_output\"],\n      [\"adversarial_stale_input.v1\", \"stale_input\"],\n    ]);\n    const roleCounts = { target_failure: 0, prior_success: 0, boundary: 0, adversarial: 0 };\n    const recordedSources = new Set();\n    const datasetCaseIDs = new Set();\n    for (const entry of dataset.cases) {\n      const at = `${where} dataset case ${entry.id}`;\n      if (datasetCaseIDs.has(entry.id)) throw new Error(`${at} appears twice in the dataset`);\n      datasetCaseIDs.add(entry.id);\n      if (!unitsByID.has(entry.source_unit_id)) throw new Error(`${at} names source unit ${entry.source_unit_id} that is not an analysis unit of this report`);\n      if (!familyUnits.has(entry.source_unit_id)) throw new Error(`${at} names source unit ${entry.source_unit_id} that is not a member of task family ${family.id}`);\n      if (generatorPerturbation.get(entry.generator) !== entry.perturbation) {\n        throw new Error(`${at} generator ${entry.generator} disagrees with perturbation ${entry.perturbation}`);\n      }\n      if (entry.perturbation === \"none\" && entry.id !== entry.source_unit_id) {\n        throw new Error(`${at} replays a recorded flow but is not identified by its source unit`);\n      }\n      if (entry.perturbation !== \"none\" && entry.id !== `${entry.generator}:${entry.source_unit_id}`) {\n        throw new Error(`${at} is a generated case whose id does not name its generator and source unit`);\n      }\n      roleCounts[entry.role] += 1;\n      if (entry.role === \"target_failure\" || entry.role === \"prior_success\") recordedSources.add(entry.source_unit_id);\n    }\n    for (const entry of dataset.cases) {\n      // Leakage: a generated case must not perturb a unit the manifest already\n      // replays unperturbed.\n      if (entry.perturbation !== \"none\" && recordedSources.has(entry.source_unit_id)) {\n        throw new Error(`${where} generated case ${entry.id} perturbs a unit the dataset already replays unperturbed`);\n      }\n    }\n","sourceCodeStart":246,"sourceCodeEnd":282,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/packages/shared/contracts/scripts/validate-continuous-improvement.mjs#L246-L282","documentation":"Thrown when a dataset case's generator and perturbation disagree per the fixed mapping: recorded_case.v1→none, boundary_guard_threshold.v1→guard_threshold_boundary, adversarial_misleading_tool_output.v1→misleading_tool_output, adversarial_stale_input.v1→stale_input. Note that generatorPerturbation.get on an UNKNOWN generator returns undefined, so an unrecognized generator string also triggers this error.","triggerScenarios":"Pairing e.g. generator \"recorded_case.v1\" with perturbation \"stale_input\"; introducing a new generator id not in the map; misspelling either field.","commonSituations":"Adding a new perturbation type to the pipeline without extending the validator's map; hand-editing entries; version skew between the composer that builds datasets and the validator.","solutions":["Set generator and perturbation to a matched pair from the map (e.g. boundary_guard_threshold.v1 with guard_threshold_boundary).","If the generator name is unrecognized, correct it to one of the four known generators or extend generatorPerturbation in lockstep with the composer.","Re-run the validator; a mismatch here usually means the dataset composer and validator drifted, so check both."],"exampleFix":"// before\n{generator: \"boundary_guard_threshold.v1\", perturbation: \"none\", ...}\n\n// after\n{generator: \"boundary_guard_threshold.v1\", perturbation: \"guard_threshold_boundary\", ...}","handlingStrategy":"validation","validationCode":"const GEN_PERT = new Map([\n  [\"recorded_case.v1\", \"none\"],\n  [\"boundary_guard_threshold.v1\", \"guard_threshold_boundary\"],\n  [\"adversarial_misleading_tool_output.v1\", \"misleading_tool_output\"],\n  [\"adversarial_stale_input.v1\", \"stale_input\"],\n]);\nconst pairsOk = dataset.cases.every((c) => GEN_PERT.get(c.generator) === c.perturbation);","typeGuard":"const GENERATORS = [\"recorded_case.v1\", \"boundary_guard_threshold.v1\", \"adversarial_misleading_tool_output.v1\", \"adversarial_stale_input.v1\"];\nfunction isKnownGenerator(g) { return GENERATORS.includes(g); }","tryCatchPattern":null,"preventionTips":["Never set generator and perturbation independently — store one and look up the other from the shared map.","When adding a new generator, update the composer, the validator map and any fixtures in the same commit."],"tags":["validation","dataset","generator-mapping"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}