JuliusBrussee/caveman · error · Error

${where} dataset carries ${roleCounts.boundary} boundary cas

Error message

${where} dataset carries ${roleCounts.boundary} boundary cases, want at most one

What it means

Thrown when more than one dataset case carries role "boundary". The dataset composition allows at most one boundary (guard-threshold) case; a second would double-count the same threshold probe as extra coverage.

Source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:305

    // a confidence guard reads. A ChangeSet whose applicability never reads that
    // evidence has no threshold to sit beside, so those two cases must NOT be
    // generated for it — a "boundary" case against a guard that does not exist
    // tests nothing while counting as adversarial coverage.
    const confidenceGuard = item.change_set.applicability.all.some((condition) =>
      condition === "failure_location_confidence >= 0.90" ||
      condition === "symbol_resolution == unique" ||
      condition === "targeted_test_reproduces == true");
    const perturbations = new Set(dataset.cases.map((entry) => entry.perturbation));
    for (const [perturbation, label] of [["guard_threshold_boundary", "boundary"], ["stale_input", "stale-input"]]) {
      if (perturbations.has(perturbation) !== confidenceGuard) {
        throw new Error(`${where} ${confidenceGuard ? "omits" : "generated"} a ${label} case ${confidenceGuard ? "for" : "against"} a change set that ${confidenceGuard ? "declares" : "declares no"} confidence guard`);
      }
    }
    const expectedTargets = Math.min(4, dataset.target_failure_cases.length);
    const expectedPriors = Math.min(2, dataset.prior_success_cases.length);
    if (roleCounts.target_failure !== expectedTargets) throw new Error(`${where} dataset carries ${roleCounts.target_failure} target-failure cases, want ${expectedTargets}`);
    if (roleCounts.prior_success !== expectedPriors) throw new Error(`${where} dataset carries ${roleCounts.prior_success} prior-success cases, want ${expectedPriors}`);
    if (roleCounts.boundary > 1) throw new Error(`${where} dataset carries ${roleCounts.boundary} boundary cases, want at most one`);
    if (roleCounts.adversarial > (confidenceGuard ? 2 : 1)) throw new Error(`${where} dataset carries ${roleCounts.adversarial} adversarial cases`);
    if (roleCounts.boundary !== dataset.boundary_cases.length) throw new Error(`${where} boundary case list disagrees with the composed dataset`);
    if (roleCounts.adversarial !== dataset.generated_cases.length) throw new Error(`${where} generated case list disagrees with the composed dataset`);
    const composed = roleCounts.target_failure + roleCounts.prior_success + roleCounts.boundary + roleCounts.adversarial;
    if (composed !== dataset.cases.length) throw new Error(`${where} dataset roles (${composed}) do not account for its ${dataset.cases.length} cases`);
    if (dataset.replay_case_ids.length !== datasetCaseIDs.size) throw new Error(`${where} replay manifest carries ${dataset.replay_case_ids.length} ids for ${datasetCaseIDs.size} dataset cases`);
    for (const datasetCaseID of dataset.replay_case_ids) {
      if (!datasetCaseIDs.has(datasetCaseID)) throw new Error(`${where} replay manifest case ${datasetCaseID} is not a composed dataset case`);
    }

    // The guard grader exists exactly when there is a perturbed case to catch a
    // candidate on; a required grader with no case behind it is decoration.
    const perturbed = dataset.cases.some((entry) => entry.perturbation !== "none");
    if (perturbed !== item.eval_pack.graders.includes("guard_respected")) {
      throw new Error(`${where} guard_respected grader ${perturbed ? "missing for" : "declared without"} perturbed dataset cases`);
    }
    for (const proof of item.replay.trial_proofs) {
      if (!datasetCaseIDs.has(proof.dataset_case_id)) throw new Error(`${where} replay trial ${proof.id} replays a case outside the composed dataset`);

View on GitHub (pinned to 27d5a3981a)

Solutions

Keep exactly one boundary case (and note error 495's rule: with a confidence guard present, exactly the guard_threshold_boundary perturbation must exist; with none, zero).
Delete the surplus boundary entries and their ids from replay_case_ids / boundary_cases.
Deduplicate by perturbation type in the composer so only one boundary case can be emitted.

Example fix

// before
[{...perturbation:"guard_threshold_boundary", role:"boundary"},
 {...perturbation:"guard_threshold_boundary", role:"boundary"}]

// after
[{...perturbation:"guard_threshold_boundary", role:"boundary"}]

Defensive patterns

Strategy: validation

Validate before calling

const boundaryCount = dataset.cases.filter((c) => c.role === "boundary").length;
const ok = boundaryCount <= 1;

Prevention

Emit at most one boundary case per dataset in the generator (deduplicate by perturbation type).
When merging datasets, drop duplicate boundary entries before writing the manifest.

When it happens

Trigger: Two cases generated with generator boundary_guard_threshold.v1 (or hand-assigned role "boundary") for the same change set.

Common situations: Running the boundary generator twice during dataset assembly; merging dataset drafts that each contributed a boundary case.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/28ffa341bc901977. Report an issue: GitHub.