JuliusBrussee/caveman · error · Error

${where} is diagnosis-only but carries a change set, eval pa

Error message

${where} is diagnosis-only but carries a change set, eval pack or replay

What it means

Thrown when a case has status "diagnosis_only" but still carries proposal artifacts: a non-empty change_set.operator_id, a non-empty eval_pack.id, or a replay.status other than "not_run". An abstaining case must carry nothing that reads as a proposed change; otherwise the report presents a non-proposal as a proposal.

Source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:217

    for (const edge of slice.unproven_adjacencies) {
      // The whole point of the separate list: nothing in it may read as proven.
      if (provenDependency(edge)) throw new Error(`${where} unproven adjacency ${edge.from}->${edge.to} is in fact a proven hard dependency`);
      if (slice.hard_edges.some((hard) => hard.from === edge.from && hard.to === edge.to && hard.type === edge.type)) {
        throw new Error(`${where} adjacency ${edge.from}->${edge.to} appears in both the hard chain and the unproven list`);
      }
    }
    const upstream = new Set(item.diagnosis.root_cause_hypothesis.upstream_node_ids);
    for (const nodeID of upstream) {
      if (!slice.node_chain.includes(nodeID) || nodeID === slice.manifestation_node_id) {
        throw new Error(`${where} diagnosis upstream node ${nodeID} is not an upstream node of its causal slice`);
      }
    }

    // A case that proposes nothing must carry nothing: no operator, no eval
    // pack, no replay. Anything else would present an abstention as a proposal.
    if (item.status === "diagnosis_only") {
      if (item.change_set.operator_id !== "" || item.eval_pack.id !== "" || item.replay.status !== "not_run") {
        throw new Error(`${where} is diagnosis-only but carries a change set, eval pack or replay`);
      }
      continue;
    }
    if (item.change_set.case_id !== item.id || item.eval_pack.case_id !== item.id) {
      throw new Error(`${where} change set or eval pack is bound to another case`);
    }

    // The case must investigate the pair its own opportunity named. A cohort
    // built from a differently-derived pair would describe one comparison and
    // measure another.
    const source = report.opportunities.find((opportunity) => opportunity.id === item.opportunity_id);
    if (!source) throw new Error(`${where} names an opportunity that is not in this report`);
    if (armVariants.get("baseline").id !== source.current_variant_id || armVariants.get("alternative").id !== source.alternative_variant_id) {
      throw new Error(`${where} cohort arms (${armVariants.get("baseline").id}/${armVariants.get("alternative").id}) are not the variants opportunity ${source.id} named`);
    }

    // Every required grader is one the recorded interpreter actually computes.
    // `json_schema` is deliberately absent: over a typed interpreter result it

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Reset the proposal fields: change_set.operator_id = "", eval_pack.id = "", replay.status = "not_run" (and empty the rest of those sub-objects per the contract).
2. If the case actually proposes a change, change status back to the proposal status instead of clearing fields.
3. Re-run the validator; this branch `continue`s, so the rest of the case is skipped once clean.

Example fix

// before
status: "diagnosis_only"
change_set: {operator_id: "op-7", ...}
eval_pack: {id: "ep-7", ...}
replay: {status: "passed"}

// after
status: "diagnosis_only"
change_set: {operator_id: "", ...}
eval_pack: {id: "", ...}
replay: {status: "not_run"}

const abstentionClean = item.status !== "diagnosis_only" || (
  item.change_set.operator_id === "" && item.eval_pack.id === "" && item.replay.status === "not_run"
);

function isCleanDiagnosisOnly(item) {
  return item.status !== "diagnosis_only" ||
    (item.change_set.operator_id === "" && item.eval_pack.id === "" && item.replay.status === "not_run");
}

When it happens

Trigger: Setting status to "diagnosis_only" on a case that was originally a full proposal case, without clearing the change set, eval pack and replay fields back to their empty/not_run defaults.

Common situations: Downgrading a case after its proposal was rejected; templating new cases from full cases and only editing the status field.

Related errors

• ${where} hard chain edge ${edge.from}->${edge.to} is not a h
• ${where} change set or eval pack is bound to another case
• {label}: duplicate identity {'/'.join(identity)}
• {label}: pricing changed without a new verified_at snapshot
• cachebench: trace request %q has invalid identity, body, or

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/ca2aa695b6a706c5. Report an issue: GitHub.