JuliusBrussee/caveman · error · Error

is a safety finding carrying expected value

Error message

${where} is a safety finding carrying expected value ${opportunity.expected_value}

What it means

A safety finding must not attach a dollar-denominated expected value: the validator requires opportunity.expected_value === 0 when type === "safety" and throws with the offending value otherwise. The rationale (in the script's comments) is that copying an efficiency finding's expected value would let the same money be counted twice under two detectors, so hazard reports carry no financial claim.

Solutions

1. Set expected_value to 0 on every opportunity whose type is "safety".
2. Fix the emitting detector so it only computes expected_value for efficiency-type opportunities.
3. If the finding truly quantifies savings, emit it as an efficiency opportunity instead of safety.
4. Re-run the validator to confirm safety findings carry zero expected value.

Example fix

// before
{ "id": "opp-3", "type": "safety", "expected_value": 142.5 }
// after
{ "id": "opp-3", "type": "safety", "expected_value": 0 }

if (report.opportunities.some((o) => o.type === "safety" && o.expected_value !== 0)) {
  throw new Error("safety finding carries non-zero expected_value");
}

const carriesNoMoney = (o) => o.type !== "safety" || o.expected_value === 0;

try {
  await runValidator([reportPath, spansPath]);
} catch (err) {
  if (String(err.message).includes("carrying expected value")) {
    console.error("Zero expected_value on safety findings; money belongs to efficiency findings only.");
  }
  throw err;
}

When it happens

Trigger: Running the validator on a report where an opportunity with type === "safety" has expected_value set to any non-zero number (e.g. a cost-saving estimate inherited from an efficiency finding).

Common situations: Detectors computing expected_value uniformly for all opportunity types; fixtures regenerated after a detector was reclassified from efficiency to safety; template reuse in report generators.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• is a safety finding with an alternative variant
• compares two workflows without naming both variants
• is a safety finding carrying another workflow's metrics
• is not a workflow variant of this report
• canonical span

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/cd7866b358bce97f. Report an issue: GitHub.

Appendix: source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:69

  // from, and a safety finding carries no borrowed dollar figure: copying the
  // efficiency finding's alternative metrics and expected value would let the
  // same money be counted twice under two detectors.
  const variantIds = new Set(report.workflow_variants.map((variant) => variant.id));
  for (const opportunity of report.opportunities) {
    const where = `report fixture ${reportPaths[index]}: opportunity ${opportunity.id}`;
    for (const key of ["current_variant_id", "alternative_variant_id"]) {
      if (opportunity[key] && !variantIds.has(opportunity[key])) {
        throw new Error(`${where} ${key} ${opportunity[key]} is not a workflow variant of this report`);
      }
    }
    if (opportunity.detector_id === "dominated-workflow") {
      if (!opportunity.current_variant_id || !opportunity.alternative_variant_id) {
        throw new Error(`${where} compares two workflows without naming both variants`);
      }
    }
    if (opportunity.type === "safety") {
      if (opportunity.alternative_variant_id) throw new Error(`${where} is a safety finding with an alternative variant`);
      if (opportunity.expected_value !== 0) throw new Error(`${where} is a safety finding carrying expected value ${opportunity.expected_value}`);
      if (opportunity.alternative_metrics.cost_per_outcome_usd !== null || opportunity.alternative_metrics.eligible_runs !== 0) {
        throw new Error(`${where} is a safety finding carrying another workflow's metrics`);
      }
    }
  }

  // A relationship is a count, so it must be recomputable from the counts it
  // carries. Anything a reader cannot re-derive is a claim, not evidence.
  const themeIds = new Set(report.themes.map((theme) => theme.id));
  for (const relationship of report.relationships) {
    const where = `report fixture ${reportPaths[index]}: relationship ${relationship.id}`;
    if (!themeIds.has(relationship.theme_a_id) || !themeIds.has(relationship.theme_b_id)) {
      throw new Error(`${where} references a theme that is not in this report`);
    }
    if (!(relationship.theme_a_id < relationship.theme_b_id)) {
      throw new Error(`${where} theme ids are not in canonical order`);
    }
    if (relationship.shared_unit_count > relationship.theme_a_unit_count || relationship.shared_unit_count > relationship.theme_b_unit_count) {

View on GitHub (pinned to 3ee70a1026)