JuliusBrussee/caveman · error · Error

${where} representative trace ${trace.analysis_unit_id} is n

Error message

${where} representative trace ${trace.analysis_unit_id} is not a member of the cohort's task family

What it means

Stronger than a plain existence check: the trace's unit must also be a member of the cohort's task family, i.e. appear in that family's analysis_unit_ids (validate-continuous-improvement.mjs:163). A trace from a unit of another family cannot speak to this cohort's comparison even though it is a valid analysis unit of the report.

Source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:163

    let comparedUnits = 0;
    for (const arm of item.cohort.arms) {
      const variant = variantsByID.get(arm.variant_id);
      if (!variant) throw new Error(`${where} cohort arm ${arm.role} references a workflow variant that is not in this report`);
      if (variant.task_family_id !== family.id) throw new Error(`${where} cohort arm ${arm.role} uses a variant of another task family`);
      armVariants.set(arm.role, variant);
      comparedUnits += arm.unit_count;
    }
    const excluded = item.cohort.excluded_units.reduce((total, exclusion) => total + exclusion.unit_count, 0);
    if (comparedUnits + excluded !== item.cohort.family_unit_count) {
      throw new Error(`${where} cohort arms (${comparedUnits}) plus exclusions (${excluded}) do not account for its ${item.cohort.family_unit_count} family units`);
    }

    const selectedPerArm = new Map();
    for (const trace of item.representative_traces) {
      const variant = armVariants.get(trace.arm);
      if (!variant || variant.id !== trace.variant_id) throw new Error(`${where} representative trace names a variant that is not that arm's variant`);
      if (!unitsByID.has(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} is not an analysis unit of this report`);
      if (!familyUnits.has(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} is not a member of the cohort's task family`);
      if (!variant.analysis_unit_ids.includes(trace.analysis_unit_id)) throw new Error(`${where} representative trace ${trace.analysis_unit_id} did not run that arm's workflow variant`);
      selectedPerArm.set(trace.arm, (selectedPerArm.get(trace.arm) ?? 0) + 1);
    }
    for (const [arm, count] of selectedPerArm) {
      if (count > 3) throw new Error(`${where} arm ${arm} carries ${count} representative traces, more than three`);
    }

    const slice = item.causal_slice;
    const sliceVariant = variantsByID.get(slice.variant_id);
    if (!sliceVariant) throw new Error(`${where} causal slice references a workflow variant that is not in this report`);
    if (sliceVariant.id !== armVariants.get("baseline").id) throw new Error(`${where} causal slice is not taken from the baseline arm's variant`);
    const sliceNodes = new Set(sliceVariant.nodes.map((node) => node.id));
    if (slice.manifestation_node_id !== "" && !sliceNodes.has(slice.manifestation_node_id)) {
      throw new Error(`${where} manifestation node ${slice.manifestation_node_id} is not a node of variant ${sliceVariant.id}`);
    }
    if (item.diagnosis.manifestation.node_id !== slice.manifestation_node_id) {
      throw new Error(`${where} diagnosis manifestation node disagrees with its causal slice`);
    }

View on GitHub (pinned to 766dce6b13)

Solutions

1. Choose a trace whose analysis_unit_id is listed in the cohort family's analysis_unit_ids.
2. If the unit's membership was mislabeled, fix family.analysis_unit_ids — then re-check errors 603 and 611 for the same family.
3. Re-run the validator.

Example fix

// before — cohort family is family-codegen, but the unit belongs to family-doc
{ "arm": "baseline", "variant_id": "wf-build-01", "analysis_unit_id": "unit-doc-3" }
// after
{ "arm": "baseline", "variant_id": "wf-build-01", "analysis_unit_id": "unit-code-1" }

const foreignFamilyTraces = (report) => {
  const byID = new Map(report.task_families.map((f) => [f.id, f]));
  return report.cases.flatMap((c) => {
    const units = new Set(byID.get(c.cohort.task_family_id)?.analysis_unit_ids ?? []);
    return c.representative_traces
      .filter((t) => !units.has(t.analysis_unit_id)).map((t) => `${c.id}:${t.analysis_unit_id}`);
  });
};

try {
  execFileSync(process.execPath, [VALIDATOR, reportPath, spansPath]);
} catch (err) {
  if (/not a member of the cohort's task family/.test(err.message)) {
    failCI(`trace unit belongs to another family: ${err.message}`);
  } else throw err;
}

When it happens

Trigger: The trace reuses a unit from a sibling family (unit-doc-3 while the cohort's family is family-codegen); the unit was moved between families; family.analysis_unit_ids was edited without re-selecting traces.

Common situations: Multi-family fixtures where unit ids look interchangeable; reorganizing family membership during a refactor; copy-pasting a trace entry from a case about a different task family.

Related errors

• ${where} representative trace ${trace.analysis_unit_id} did
• ${where} cohort family unit count disagrees with the task fa
• ${where} cohort arm ${arm.role} uses a variant of another ta
• ${where} representative trace names a variant that is not th
• ${where} representative trace ${trace.analysis_unit_id} is n

AI-assisted analysis of JuliusBrussee/caveman@766dce6b13 (2026-08-18). Data as JSON: /api/errors/2510172331237de1. Report an issue: GitHub.