JuliusBrussee/caveman · error · Error

${where} ${confidenceGuard ? "omits" : "generated"} a ${labe

Error message

${where} ${confidenceGuard ? "omits" : "generated"} a ${label} case ${confidenceGuard ? "for" : "against"} a change set that ${confidenceGuard ? "declares" : "declares no"} confidence guard

What it means

Thrown when the presence of a boundary (guard_threshold_boundary) or stale-input case in the dataset does not match whether the change set's applicability declares a confidence guard (one of the exact conditions "failure_location_confidence >= 0.90", "symbol_resolution == unique", "targeted_test_reproduces == true"). Those two generators perturb localization evidence a guard reads; with no guard there is no threshold to test, and with a guard present omitting them loses the coverage that matters most.

Source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:298

      }
    }

    // Manifest composition arithmetic (spec 18.2): target-failure + prior-success
    // + one boundary + up to two adversarial, each capped and each recomputable.
    //
    // The boundary and stale-input generators perturb the localization evidence
    // a confidence guard reads. A ChangeSet whose applicability never reads that
    // evidence has no threshold to sit beside, so those two cases must NOT be
    // generated for it — a "boundary" case against a guard that does not exist
    // tests nothing while counting as adversarial coverage.
    const confidenceGuard = item.change_set.applicability.all.some((condition) =>
      condition === "failure_location_confidence >= 0.90" ||
      condition === "symbol_resolution == unique" ||
      condition === "targeted_test_reproduces == true");
    const perturbations = new Set(dataset.cases.map((entry) => entry.perturbation));
    for (const [perturbation, label] of [["guard_threshold_boundary", "boundary"], ["stale_input", "stale-input"]]) {
      if (perturbations.has(perturbation) !== confidenceGuard) {
        throw new Error(`${where} ${confidenceGuard ? "omits" : "generated"} a ${label} case ${confidenceGuard ? "for" : "against"} a change set that ${confidenceGuard ? "declares" : "declares no"} confidence guard`);
      }
    }
    const expectedTargets = Math.min(4, dataset.target_failure_cases.length);
    const expectedPriors = Math.min(2, dataset.prior_success_cases.length);
    if (roleCounts.target_failure !== expectedTargets) throw new Error(`${where} dataset carries ${roleCounts.target_failure} target-failure cases, want ${expectedTargets}`);
    if (roleCounts.prior_success !== expectedPriors) throw new Error(`${where} dataset carries ${roleCounts.prior_success} prior-success cases, want ${expectedPriors}`);
    if (roleCounts.boundary > 1) throw new Error(`${where} dataset carries ${roleCounts.boundary} boundary cases, want at most one`);
    if (roleCounts.adversarial > (confidenceGuard ? 2 : 1)) throw new Error(`${where} dataset carries ${roleCounts.adversarial} adversarial cases`);
    if (roleCounts.boundary !== dataset.boundary_cases.length) throw new Error(`${where} boundary case list disagrees with the composed dataset`);
    if (roleCounts.adversarial !== dataset.generated_cases.length) throw new Error(`${where} generated case list disagrees with the composed dataset`);
    const composed = roleCounts.target_failure + roleCounts.prior_success + roleCounts.boundary + roleCounts.adversarial;
    if (composed !== dataset.cases.length) throw new Error(`${where} dataset roles (${composed}) do not account for its ${dataset.cases.length} cases`);
    if (dataset.replay_case_ids.length !== datasetCaseIDs.size) throw new Error(`${where} replay manifest carries ${dataset.replay_case_ids.length} ids for ${datasetCaseIDs.size} dataset cases`);
    for (const datasetCaseID of dataset.replay_case_ids) {
      if (!datasetCaseIDs.has(datasetCaseID)) throw new Error(`${where} replay manifest case ${datasetCaseID} is not a composed dataset case`);
    }

    // The guard grader exists exactly when there is a perturbed case to catch a

View on GitHub (pinned to 27d5a3981a)

Solutions

1. If the change set declares a confidence guard (exact string match), ensure the dataset contains both a guard_threshold_boundary case and a stale_input case.
2. If it declares no guard, remove boundary and stale-input cases from the dataset.
3. If you believe a condition IS a guard but is worded differently, align the string to one of the three recognized conditions (the check is exact-match, not semantic).

Example fix

// before
applicability.all: []            // no guard
dataset includes {generator:"boundary_guard_threshold.v1", perturbation:"guard_threshold_boundary"}

// after — either drop the case, or declare the guard
applicability.all: ["failure_location_confidence >= 0.90"]

const GUARD_CONDITIONS = new Set([
  "failure_location_confidence >= 0.90",
  "symbol_resolution == unique",
  "targeted_test_reproduces == true",
]);
const hasGuard = item.change_set.applicability.all.some((c) => GUARD_CONDITIONS.has(c));
const perts = new Set(dataset.cases.map((c) => c.perturbation));
const aligned = perts.has("guard_threshold_boundary") === hasGuard && perts.has("stale_input") === hasGuard;

When it happens

Trigger: A change set whose applicability.all contains none of the three guard conditions but whose dataset includes a boundary or stale-input case; conversely, a guarded change set whose dataset lacks them.

Common situations: Editing applicability conditions (rewording or removing a guard) without recomposing the dataset; reusing one dataset template for all cases regardless of applicability.

Related errors

• ${at} appears twice in the dataset
• ${at} names source unit ${entry.source_unit_id} that is not
• ${at} generator ${entry.generator} disagrees with perturbati
• ${at} replays a recorded flow but is not identified by its s
• ${at} is a generated case whose id does not name its generat

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/0daab5805573fac0. Report an issue: GitHub.