JuliusBrussee/caveman · error · Error

${where} representative trace ${trace.analysis_unit_id} is n

Error message

${where} representative trace ${trace.analysis_unit_id} is not an analysis unit of this report

What it means

A representative trace's analysis_unit_id must resolve in unitsByID, the map of report.analysis_units (validate-continuous-improvement.mjs:162). Representative traces are evidence drawn from the report's own population, so a unit id that exists nowhere in this report — a typo, or a unit defined only in a sibling fixture — is rejected.

Source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:162

    const armVariants = new Map();
    let comparedUnits = 0;
    for (const arm of item.cohort.arms) {
      const variant = variantsByID.get(arm.variant_id);
      if (!variant) throw new Error(`${where} cohort arm ${arm.role} references a workflow variant that is not in this report`);
      if (variant.task_family_id !== family.id) throw new Error(`${where} cohort arm ${arm.role} uses a variant of another task family`);
      armVariants.set(arm.role, variant);
      comparedUnits += arm.unit_count;
    }
    const excluded = item.cohort.excluded_units.reduce((total, exclusion) => total + exclusion.unit_count, 0);
    if (comparedUnits + excluded !== item.cohort.family_unit_count) {
      throw new Error(`${where} cohort arms (${comparedUnits}) plus exclusions (${excluded}) do not account for its ${item.cohort.family_unit_count} family units`);
    }

    const selectedPerArm = new Map();
    for (const trace of item.representative_traces) {
      const variant = armVariants.get(trace.arm);
      if (!variant || variant.id !== trace.variant_id) throw new Error(`${where} representative trace names a variant that is not that arm's variant`);
      if (!unitsByID.has(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} is not an analysis unit of this report`);
      if (!familyUnits.has(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} is not a member of the cohort's task family`);
      if (!variant.analysis_unit_ids.includes(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} did not run that arm's workflow variant`);
      selectedPerArm.set(trace.arm, (selectedPerArm.get(trace.arm) ?? 0) + 1);
    }
    for (const [arm, count] of selectedPerArm) {
      if (count > 3) throw new Error(`${where} arm ${arm} carries ${count} representative traces, more than three`);
    }

    const slice = item.causal_slice;
    const sliceVariant = variantsByID.get(slice.variant_id);
    if (!sliceVariant) throw new Error(`${where} causal slice references a workflow variant that is not in this report`);
    if (sliceVariant.id !== armVariants.get("baseline").id) throw new Error(`${where} causal slice is not taken from the baseline arm's variant`);
    const sliceNodes = new Set(sliceVariant.nodes.map((node) => node.id));
    if (slice.manifestation_node_id !== "" && !sliceNodes.has(slice.manifestation_node_id)) {
      throw new Error(`${where} manifestation node ${slice.manifestation_node_id} is not a node of variant ${sliceVariant.id}`);
    }
    if (item.diagnosis.manifestation.node_id !== slice.manifestation_node_id) {
      throw new Error(`${where} diagnosis manifestation node disagrees with its causal slice`);

View on GitHub (pinned to 766dce6b13)

Solutions

  1. Restore or correct the unit id so it matches an entry in this report's analysis_units.
  2. If the unit was pruned, either put it back or drop the trace that cites it.
  3. Keep each case's traces and the units they cite in the same report file, then re-run the validator.

Example fix

// before — report defines analysis unit "unit-901"
{ "arm": "baseline", "variant_id": "wf-build-01", "analysis_unit_id": "unit-091" }
// after
{ "arm": "baseline", "variant_id": "wf-build-01", "analysis_unit_id": "unit-901" }
Defensive patterns

Strategy: validation

Validate before calling

const danglingTraceUnits = (report) => {
  const ids = new Set(report.analysis_units.map((u) => u.id));
  return report.cases.flatMap((c) =>
    c.representative_traces.filter((t) => !ids.has(t.analysis_unit_id)).map((t) => `${c.id}:${t.analysis_unit_id}`));
};

Try / catch

try {
  execFileSync(process.execPath, [VALIDATOR, reportPath, spansPath]);
} catch (err) {
  if (/not an analysis unit of this report/.test(err.message)) {
    failCI(`trace cites a unit outside this report: ${err.message}`);
  } else throw err;
}

Prevention

When it happens

Trigger: trace.analysis_unit_id is a typo (unit-091 vs unit-901); the analysis_units entry was pruned while its trace was kept; the fixture was split across files, leaving the unit in another report.

Common situations: Renumbering unit ids when reorganizing fixtures; trimming the analysis_units array to shrink a file; generators that emit traces from a unit list that was later filtered.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/db12aaeb99b61b9f. Report an issue: GitHub.