JuliusBrussee/caveman · error · Error

references a theme that is not in this report

Error message

${where} references a theme that is not in this report

What it means

Each relationship is an edge between exactly two themes of the same report, so the validator builds a Set of report.themes ids and throws if either relationship.theme_a_id or relationship.theme_b_id is missing from it. This referential-integrity check ensures every relationship can be recomputed from evidence the report itself carries.

Solutions

  1. Fix relationship.theme_a_id / theme_b_id to ids present in report.themes.
  2. Delete relationships whose themes were removed from the report (and re-check the relationship count stays non-zero across fixtures).
  3. If the theme belongs in the report, re-add it to report.themes rather than leaving a dangling edge.
  4. Re-run the validator to confirm all relationship endpoints resolve.

Example fix

// before
{ "id": "rel-1", "theme_a_id": "theme-retry-old", "theme_b_id": "theme-guard" }
// after
{ "id": "rel-1", "theme_a_id": "theme-retry", "theme_b_id": "theme-guard" }
Defensive patterns

Strategy: validation

Validate before calling

const themeIds = new Set(report.themes.map((t) => t.id));
const dangling = report.relationships.filter((r) => !themeIds.has(r.theme_a_id) || !themeIds.has(r.theme_b_id));
if (dangling.length) throw new Error(`relationships reference unknown themes: ${dangling.map((r) => r.id)}`);

Type guard

const endpointsResolved = (r, report) => { const ids = new Set(report.themes.map((t) => t.id)); return ids.has(r.theme_a_id) && ids.has(r.theme_b_id); };

Try / catch

try {
  await runValidator([reportPath, spansPath]);
} catch (err) {
  if (String(err.message).includes("references a theme that is not in this report")) {
    console.error("Drop or re-point relationships whose themes are absent from report.themes.");
  }
  throw err;
}

Prevention

When it happens

Trigger: Running the validator on a report where a relationship references a theme id not present in report.themes — e.g. after a theme was dropped from the report or its id renamed.

Common situations: Pruning low-support themes without pruning their relationships; theme-id renames during fixture regeneration; merging relationships from a different report's theme set.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/a534ff4da5871c31. Report an issue: GitHub.

Appendix: source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:82

        throw new Error(`${where} compares two workflows without naming both variants`);
      }
    }
    if (opportunity.type === "safety") {
      if (opportunity.alternative_variant_id) throw new Error(`${where} is a safety finding with an alternative variant`);
      if (opportunity.expected_value !== 0) throw new Error(`${where} is a safety finding carrying expected value ${opportunity.expected_value}`);
      if (opportunity.alternative_metrics.cost_per_outcome_usd !== null || opportunity.alternative_metrics.eligible_runs !== 0) {
        throw new Error(`${where} is a safety finding carrying another workflow's metrics`);
      }
    }
  }

  // A relationship is a count, so it must be recomputable from the counts it
  // carries. Anything a reader cannot re-derive is a claim, not evidence.
  const themeIds = new Set(report.themes.map((theme) => theme.id));
  for (const relationship of report.relationships) {
    const where = `report fixture ${reportPaths[index]}: relationship ${relationship.id}`;
    if (!themeIds.has(relationship.theme_a_id) || !themeIds.has(relationship.theme_b_id)) {
      throw new Error(`${where} references a theme that is not in this report`);
    }
    if (!(relationship.theme_a_id < relationship.theme_b_id)) {
      throw new Error(`${where} theme ids are not in canonical order`);
    }
    if (relationship.shared_unit_count > relationship.theme_a_unit_count || relationship.shared_unit_count > relationship.theme_b_unit_count) {
      throw new Error(`${where} shares more units than either theme has`);
    }
    if (relationship.shared_unit_ids.length > relationship.shared_unit_count) {
      throw new Error(`${where} carries more shared unit ids than its shared unit count`);
    }
    const expectedBGivenA = relationship.shared_unit_count / relationship.theme_a_unit_count;
    const expectedAGivenB = relationship.shared_unit_count / relationship.theme_b_unit_count;
    if (Math.abs(relationship.probability_b_given_a - expectedBGivenA) > 1e-9) {
      throw new Error(`${where} probability_b_given_a ${relationship.probability_b_given_a} != ${expectedBGivenA}`);
    }
    if (Math.abs(relationship.probability_a_given_b - expectedAGivenB) > 1e-9) {
      throw new Error(`${where} probability_a_given_b ${relationship.probability_a_given_b} != ${expectedAGivenB}`);
    }

View on GitHub (pinned to 3ee70a1026)