JuliusBrussee/caveman · error · Error

theme ids are not in canonical order

Error message

${where} theme ids are not in canonical order

What it means

Relationship endpoints must be stored in a canonical order: the validator requires relationship.theme_a_id < relationship.theme_b_id (lexicographic string comparison) so the same pair of themes always serializes one way and is not double-counted as two distinct relationships. It throws when the ids are equal or stored in descending order.

Solutions

  1. Swap the two ids so theme_a_id is the lexicographically smaller one (keeping counts/probabilities attached to the correct endpoints).
  2. Sort theme ids when emitting relationships in the report generator so ordering is deterministic.
  3. Deduplicate reversed pairs in the generator before writing the report.
  4. Re-run the validator to confirm canonical ordering.

Example fix

// before
{ "theme_a_id": "theme-retry", "theme_b_id": "theme-guard" }
// after
{ "theme_a_id": "theme-guard", "theme_b_id": "theme-retry" }
Defensive patterns

Strategy: validation

Validate before calling

if (report.relationships.some((r) => !(r.theme_a_id < r.theme_b_id))) {
  throw new Error("relationship endpoints not in canonical order");
}

Type guard

const isCanonicallyOrdered = (r) => r.theme_a_id < r.theme_b_id;

Try / catch

try {
  await runValidator([reportPath, spansPath]);
} catch (err) {
  if (String(err.message).includes("theme ids are not in canonical order")) {
    console.error("Sort the pair lexicographically and swap counts accordingly.");
  }
  throw err;
}

Prevention

When it happens

Trigger: Running the validator on a report where a relationship has theme_a_id >= theme_b_id, e.g. {"theme_a_id": "theme-guard", "theme_b_id": "theme-retry"} with "theme-retry" < "theme-guard", or both fields equal.

Common situations: Relationship emitted while iterating themes in discovery order instead of sorted order; deduplication logic absent so a pair appears as both (a,b) and (b,a); hand-written fixtures.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/6e6a1e41f17de10c. Report an issue: GitHub.

Appendix: source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:85

    if (opportunity.type === "safety") {
      if (opportunity.alternative_variant_id) throw new Error(`${where} is a safety finding with an alternative variant`);
      if (opportunity.expected_value !== 0) throw new Error(`${where} is a safety finding carrying expected value ${opportunity.expected_value}`);
      if (opportunity.alternative_metrics.cost_per_outcome_usd !== null || opportunity.alternative_metrics.eligible_runs !== 0) {
        throw new Error(`${where} is a safety finding carrying another workflow's metrics`);
      }
    }
  }

  // A relationship is a count, so it must be recomputable from the counts it
  // carries. Anything a reader cannot re-derive is a claim, not evidence.
  const themeIds = new Set(report.themes.map((theme) => theme.id));
  for (const relationship of report.relationships) {
    const where = `report fixture ${reportPaths[index]}: relationship ${relationship.id}`;
    if (!themeIds.has(relationship.theme_a_id) || !themeIds.has(relationship.theme_b_id)) {
      throw new Error(`${where} references a theme that is not in this report`);
    }
    if (!(relationship.theme_a_id < relationship.theme_b_id)) {
      throw new Error(`${where} theme ids are not in canonical order`);
    }
    if (relationship.shared_unit_count > relationship.theme_a_unit_count || relationship.shared_unit_count > relationship.theme_b_unit_count) {
      throw new Error(`${where} shares more units than either theme has`);
    }
    if (relationship.shared_unit_ids.length > relationship.shared_unit_count) {
      throw new Error(`${where} carries more shared unit ids than its shared unit count`);
    }
    const expectedBGivenA = relationship.shared_unit_count / relationship.theme_a_unit_count;
    const expectedAGivenB = relationship.shared_unit_count / relationship.theme_b_unit_count;
    if (Math.abs(relationship.probability_b_given_a - expectedBGivenA) > 1e-9) {
      throw new Error(`${where} probability_b_given_a ${relationship.probability_b_given_a} != ${expectedBGivenA}`);
    }
    if (Math.abs(relationship.probability_a_given_b - expectedAGivenB) > 1e-9) {
      throw new Error(`${where} probability_a_given_b ${relationship.probability_a_given_b} != ${expectedAGivenB}`);
    }
  }
  relationshipCount += report.relationships.length;

View on GitHub (pinned to 3ee70a1026)