JuliusBrussee/caveman · error · Error

is a safety finding with an alternative variant

Error message

${where} is a safety finding with an alternative variant

What it means

A safety finding describes a hazard in the current workflow only — there is no alternative being recommended — so the validator forbids type === "safety" opportunities from carrying an alternative_variant_id. It throws when a safety finding sets that field, which would misleadingly suggest a swap is proposed. This keeps efficiency recommendations and hazard reports structurally distinguishable.

Solutions

1. Set alternative_variant_id to null (or remove the field) on the safety opportunity.
2. If an alternative workflow really is proposed, change opportunity.type to the efficiency type instead of "safety".
3. Check the detector that emitted the finding and fix it to not populate alternative_variant_id for safety findings.
4. Re-run the validator to confirm safety findings carry no alternative.

Example fix

// before
{ "id": "opp-3", "type": "safety", "current_variant_id": "variant-a", "alternative_variant_id": "variant-b" }
// after
{ "id": "opp-3", "type": "safety", "current_variant_id": "variant-a", "alternative_variant_id": null }

if (report.opportunities.some((o) => o.type === "safety" && o.alternative_variant_id)) {
  throw new Error("safety finding carries an alternative variant");
}

const isPureSafetyFinding = (o) => o.type !== "safety" || (o.alternative_variant_id ?? null) === null;

try {
  await runValidator([reportPath, spansPath]);
} catch (err) {
  if (String(err.message).includes("is a safety finding with an alternative variant")) {
    console.error("Null out alternative_variant_id or reclassify the opportunity type.");
  }
  throw err;
}

When it happens

Trigger: Running the validator on a report where an opportunity with type === "safety" has a truthy alternative_variant_id.

Common situations: Generator code copying the efficiency-opportunity template (which sets both variant ids) for safety findings; a detector reclassified from efficiency to safety without clearing the alternative variant; hand-merged fixtures.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• is a safety finding carrying expected value
• compares two workflows without naming both variants
• is a safety finding carrying another workflow's metrics
• is not a workflow variant of this report
• canonical span

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/c12eebeb1528311f. Report an issue: GitHub.

Appendix: source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:68

  // An opportunity names the exact pair of workflow variants it was emitted
  // from, and a safety finding carries no borrowed dollar figure: copying the
  // efficiency finding's alternative metrics and expected value would let the
  // same money be counted twice under two detectors.
  const variantIds = new Set(report.workflow_variants.map((variant) => variant.id));
  for (const opportunity of report.opportunities) {
    const where = `report fixture ${reportPaths[index]}: opportunity ${opportunity.id}`;
    for (const key of ["current_variant_id", "alternative_variant_id"]) {
      if (opportunity[key] && !variantIds.has(opportunity[key])) {
        throw new Error(`${where} ${key} ${opportunity[key]} is not a workflow variant of this report`);
      }
    }
    if (opportunity.detector_id === "dominated-workflow") {
      if (!opportunity.current_variant_id || !opportunity.alternative_variant_id) {
        throw new Error(`${where} compares two workflows without naming both variants`);
      }
    }
    if (opportunity.type === "safety") {
      if (opportunity.alternative_variant_id) throw new Error(`${where} is a safety finding with an alternative variant`);
      if (opportunity.expected_value !== 0) throw new Error(`${where} is a safety finding carrying expected value ${opportunity.expected_value}`);
      if (opportunity.alternative_metrics.cost_per_outcome_usd !== null || opportunity.alternative_metrics.eligible_runs !== 0) {
        throw new Error(`${where} is a safety finding carrying another workflow's metrics`);
      }
    }
  }

  // A relationship is a count, so it must be recomputable from the counts it
  // carries. Anything a reader cannot re-derive is a claim, not evidence.
  const themeIds = new Set(report.themes.map((theme) => theme.id));
  for (const relationship of report.relationships) {
    const where = `report fixture ${reportPaths[index]}: relationship ${relationship.id}`;
    if (!themeIds.has(relationship.theme_a_id) || !themeIds.has(relationship.theme_b_id)) {
      throw new Error(`${where} references a theme that is not in this report`);
    }
    if (!(relationship.theme_a_id < relationship.theme_b_id)) {
      throw new Error(`${where} theme ids are not in canonical order`);
    }

View on GitHub (pinned to 3ee70a1026)