JuliusBrussee/caveman · error · Error

is not a workflow variant of this report

Error message

${where} ${key} ${opportunity[key]} is not a workflow variant of this report

What it means

Every opportunity must reference workflow variants that actually exist in the same report: the validator builds a Set of report.workflow_variants ids and throws if an opportunity's current_variant_id or alternative_variant_id is set but not in that set. This is a referential-integrity check — an opportunity naming a variant from another report or a deleted variant is not evidence this report can back.

Solutions

1. Open the named fixture and fix opportunity.current_variant_id / alternative_variant_id to ids present in report.workflow_variants.
2. Remove or regenerate opportunities that reference variants no longer carried by the report.
3. If the variant exists in another report, move the opportunity into that report rather than cross-referencing.
4. Re-run the validator to confirm variant references resolve.

Example fix

// before
{ "id": "opp-1", "current_variant_id": "variant-a-old" }
// after
{ "id": "opp-1", "current_variant_id": "variant-a" }

const variantIds = new Set(report.workflow_variants.map((v) => v.id));
const dangling = report.opportunities.filter((o) =>
  [o.current_variant_id, o.alternative_variant_id].some((id) => id && !variantIds.has(id))
);
if (dangling.length) throw new Error(`opportunities reference unknown variants: ${dangling.map((o) => o.id)}`);

const referencesKnownVariant = (opportunity, report) => { const ids = new Set(report.workflow_variants.map((v) => v.id)); return ids.has(opportunity.current_variant_id) && (!opportunity.alternative_variant_id || ids.has(opportunity.alternative_variant_id)); };

try {
  await runValidator([reportPath, spansPath]);
} catch (err) {
  if (String(err.message).includes("is not a workflow variant of this report")) {
    console.error("Opportunity variant id is dangling — reconcile with report.workflow_variants.");
  }
  throw err;
}

When it happens

Trigger: Running the validator on a report where an opportunity has a non-empty current_variant_id or alternative_variant_id that does not match any report.workflow_variants[].id.

Common situations: Merging opportunities and variants from different report fixtures; renaming variant ids during regeneration so opportunities keep stale ids; deleting a dominated variant from workflow_variants while its opportunities remain.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

• report fixture : theme inherited a registry id that is not…
• compares two workflows without naming both variants
• is a safety finding carrying another workflow's metrics
• is a safety finding carrying expected value
• is a safety finding with an alternative variant

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/7d4e3fcf26545559. Report an issue: GitHub.

Appendix: source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:59

    }
    if (isNew !== (theme.lineage.predecessor_ids.length === 0)) {
      throw new Error(`report fixture ${reportPaths[index]}: theme ${theme.id} transition ${theme.lineage.transition} disagrees with its predecessor list`);
    }
    if (!isNew && !theme.lineage.predecessor_ids.includes(theme.registry_id)) {
      throw new Error(`report fixture ${reportPaths[index]}: theme ${theme.id} inherited a registry id that is not one of its predecessors`);
    }
  }

  // An opportunity names the exact pair of workflow variants it was emitted
  // from, and a safety finding carries no borrowed dollar figure: copying the
  // efficiency finding's alternative metrics and expected value would let the
  // same money be counted twice under two detectors.
  const variantIds = new Set(report.workflow_variants.map((variant) => variant.id));
  for (const opportunity of report.opportunities) {
    const where = `report fixture ${reportPaths[index]}: opportunity ${opportunity.id}`;
    for (const key of ["current_variant_id", "alternative_variant_id"]) {
      if (opportunity[key] && !variantIds.has(opportunity[key])) {
        throw new Error(`${where} ${key} ${opportunity[key]} is not a workflow variant of this report`);
      }
    }
    if (opportunity.detector_id === "dominated-workflow") {
      if (!opportunity.current_variant_id || !opportunity.alternative_variant_id) {
        throw new Error(`${where} compares two workflows without naming both variants`);
      }
    }
    if (opportunity.type === "safety") {
      if (opportunity.alternative_variant_id) throw new Error(`${where} is a safety finding with an alternative variant`);
      if (opportunity.expected_value !== 0) throw new Error(`${where} is a safety finding carrying expected value ${opportunity.expected_value}`);
      if (opportunity.alternative_metrics.cost_per_outcome_usd !== null || opportunity.alternative_metrics.eligible_runs !== 0) {
        throw new Error(`${where} is a safety finding carrying another workflow's metrics`);
      }
    }
  }

  // A relationship is a count, so it must be recomputable from the counts it
  // carries. Anything a reader cannot re-derive is a claim, not evidence.

View on GitHub (pinned to 3ee70a1026)