JuliusBrussee/caveman · error · Error

${at} appears twice in the dataset

Error message

${at} appears twice in the dataset

What it means

Thrown when two entries in eval_pack.dataset.cases share the same id. Dataset case ids are the join key between the composed dataset, the role lists and the replay manifest; duplicates would double-count roles and make replay_case_ids ambiguous.

Source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:259

    }

    // Every dataset case — recorded or generated — must name a real analysis
    // unit of this case's own task family, and its generator and perturbation
    // must agree. A generated case whose source unit cannot be resolved would be
    // invented evidence wearing an adversarial label.
    const dataset = item.eval_pack.dataset;
    const generatorPerturbation = new Map([
      ["recorded_case.v1", "none"],
      ["boundary_guard_threshold.v1", "guard_threshold_boundary"],
      ["adversarial_misleading_tool_output.v1", "misleading_tool_output"],
      ["adversarial_stale_input.v1", "stale_input"],
    ]);
    const roleCounts = { target_failure: 0, prior_success: 0, boundary: 0, adversarial: 0 };
    const recordedSources = new Set();
    const datasetCaseIDs = new Set();
    for (const entry of dataset.cases) {
      const at = `${where} dataset case ${entry.id}`;
      if (datasetCaseIDs.has(entry.id)) throw new Error(`${at} appears twice in the dataset`);
      datasetCaseIDs.add(entry.id);
      if (!unitsByID.has(entry.source_unit_id)) throw new Error(`${at} names source unit ${entry.source_unit_id} that is not an analysis unit of this report`);
      if (!familyUnits.has(entry.source_unit_id)) throw new Error(`${at} names source unit ${entry.source_unit_id} that is not a member of task family ${family.id}`);
      if (generatorPerturbation.get(entry.generator) !== entry.perturbation) {
        throw new Error(`${at} generator ${entry.generator} disagrees with perturbation ${entry.perturbation}`);
      }
      if (entry.perturbation === "none" && entry.id !== entry.source_unit_id) {
        throw new Error(`${at} replays a recorded flow but is not identified by its source unit`);
      }
      if (entry.perturbation !== "none" && entry.id !== `${entry.generator}:${entry.source_unit_id}`) {
        throw new Error(`${at} is a generated case whose id does not name its generator and source unit`);
      }
      roleCounts[entry.role] += 1;
      if (entry.role === "target_failure" || entry.role === "prior_success") recordedSources.add(entry.source_unit_id);
    }
    for (const entry of dataset.cases) {
      // Leakage: a generated case must not perturb a unit the manifest already
      // replays unperturbed.

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Delete the duplicate entry so every id in dataset.cases is unique.
2. If two distinct cases collided, fix their ids to follow the convention: source_unit_id for recorded (perturbation "none") cases, `${generator}:${source_unit_id}` for generated cases.
3. Audit dataset.replay_case_ids and the role lists (target_failure_cases, boundary_cases, generated_cases) for the same duplication.

Example fix

// before
dataset.cases: [{id:"u-1",...},{id:"u-1",...}]

// after
dataset.cases: [{id:"u-1",...},{id:"adversarial_stale_input.v1:u-2",...}]

const ids = dataset.cases.map((c) => c.id);
const unique = new Set(ids).size === ids.length;

When it happens

Trigger: Appending the same case entry twice during dataset assembly; two generators producing colliding ids (only possible when the id convention — generator:source_unit_id or bare source_unit_id — is violated); copy-paste of an entry with a hand-invented id.

Common situations: Manual dataset editing; merging dataset fragments from two drafts without deduplication.

Related errors

• ${where} adjacency ${edge.from}->${edge.to} appears in both
• ${at} names source unit ${entry.source_unit_id} that is not
• ${at} generator ${entry.generator} disagrees with perturbati
• ${at} replays a recorded flow but is not identified by its s
• ${at} is a generated case whose id does not name its generat

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/26a9036018da99be. Report an issue: GitHub.