JuliusBrussee/caveman · error · Error

${at} is a generated case whose id does not name its generat

Error message

${at} is a generated case whose id does not name its generator and source unit

What it means

Thrown when a generated case (perturbation != "none") has an id that is not exactly `${generator}:${source_unit_id}`. Generated-case ids must name their generator and source unit so provenance is recomputable from the id alone.

Source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:270

      ["adversarial_stale_input.v1", "stale_input"],
    ]);
    const roleCounts = { target_failure: 0, prior_success: 0, boundary: 0, adversarial: 0 };
    const recordedSources = new Set();
    const datasetCaseIDs = new Set();
    for (const entry of dataset.cases) {
      const at = `${where} dataset case ${entry.id}`;
      if (datasetCaseIDs.has(entry.id)) throw new Error(`${at} appears twice in the dataset`);
      datasetCaseIDs.add(entry.id);
      if (!unitsByID.has(entry.source_unit_id)) throw new Error(`${at} names source unit ${entry.source_unit_id} that is not an analysis unit of this report`);
      if (!familyUnits.has(entry.source_unit_id)) throw new Error(`${at} names source unit ${entry.source_unit_id} that is not a member of task family ${family.id}`);
      if (generatorPerturbation.get(entry.generator) !== entry.perturbation) {
        throw new Error(`${at} generator ${entry.generator} disagrees with perturbation ${entry.perturbation}`);
      }
      if (entry.perturbation === "none" && entry.id !== entry.source_unit_id) {
        throw new Error(`${at} replays a recorded flow but is not identified by its source unit`);
      }
      if (entry.perturbation !== "none" && entry.id !== `${entry.generator}:${entry.source_unit_id}`) {
        throw new Error(`${at} is a generated case whose id does not name its generator and source unit`);
      }
      roleCounts[entry.role] += 1;
      if (entry.role === "target_failure" || entry.role === "prior_success") recordedSources.add(entry.source_unit_id);
    }
    for (const entry of dataset.cases) {
      // Leakage: a generated case must not perturb a unit the manifest already
      // replays unperturbed.
      if (entry.perturbation !== "none" && recordedSources.has(entry.source_unit_id)) {
        throw new Error(`${where} generated case ${entry.id} perturbs a unit the dataset already replays unperturbed`);
      }
    }

    // Manifest composition arithmetic (spec 18.2): target-failure + prior-success
    // + one boundary + up to two adversarial, each capped and each recomputable.
    //
    // The boundary and stale-input generators perturb the localization evidence
    // a confidence guard reads. A ChangeSet whose applicability never reads that
    // evidence has no threshold to sit beside, so those two cases must NOT be

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Rewrite the id as `${entry.generator}:${entry.source_unit_id}` exactly (generator string first, colon, unit id).
2. After fixing, check for the leakage rule: the same source unit must not also appear as an unperturbed recorded case.
3. Keep ids machine-generated by the composer to avoid drift.

Example fix

// before
{id: "edge-case-3", generator: "adversarial_stale_input.v1", source_unit_id: "unit-07", perturbation: "stale_input"}

// after
{id: "adversarial_stale_input.v1:unit-07", generator: "adversarial_stale_input.v1", source_unit_id: "unit-07", perturbation: "stale_input"}

const generatedIdsOk = dataset.cases
  .filter((c) => c.perturbation !== "none")
  .every((c) => c.id === `${c.generator}:${c.source_unit_id}`);

When it happens

Trigger: Inventing a friendly id ("edge-case-3") for a generated case; changing the generator field without updating the id; omitting the ":" or reordering the two components.

Common situations: Hand-authoring adversarial cases; renaming generators during a version bump without rewriting ids.

Related errors

• ${at} replays a recorded flow but is not identified by its s
• ${at} appears twice in the dataset
• ${at} names source unit ${entry.source_unit_id} that is not
• ${at} generator ${entry.generator} disagrees with perturbati
• ${where} generated case ${entry.id} perturbs a unit the data

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/01835491d8dcae4f. Report an issue: GitHub.