JuliusBrussee/caveman · error · Error

canonical span

Error message

canonical span ${index}: ${ajv.errorsText(validateSpan.errors)}

What it means

This validation script for continuous-improvement contracts checks every canonical span fixture against an AJV-compiled span schema. When a span object does not conform to the schema, the script throws with the span's index in the fixture array and AJV's full error text (paths, keywords, expected types). It exists to catch malformed canonical span fixtures before they are used to validate generated improvement reports.

Solutions

1. Read the AJV errorsText in the message to find the exact dataPath and keyword violation for the reported span index
2. Open the spans fixture file and fix the span at that index to match the span schema (add/rename fields, correct types)
3. Regenerate the fixture from the canonical generator script rather than hand-editing
4. If the schema intentionally changed, update fixtures AND the span schema together so validateSpan compiles against the new contract

Example fix

// before (fixture)
{ "spanId": 42, "theme": "retry" }
// after
{ "spanId": "span-0042", "theme": "retry", "durationMs": 120 }

import Ajv2020 from 'ajv/dist/2020.js';
const validateSpan = ajv.compile(spanSchema);
const bad = spans.map((s, i) => ({ i, errors: validateSpan(s) ? null : validateSpan.errors })).filter(x => x.errors);
if (bad.length) console.error('invalid spans:', bad.map(b => `#${b.i}: ${JSON.stringify(b.errors)}`).join('\n'));

const isSpan = (s) => s !== null && typeof s === 'object' && typeof s.spanId === 'string' && typeof s.theme === 'string';

try { validateOrThrow(spans); } catch (e) { if (e.message.startsWith('canonical span')) { console.error('Fixture schema drift:', e.message); process.exitCode = 1; } else throw e; }

When it happens

Trigger: Running scripts/validate-continuous-improvement.mjs when any element of the loaded spans fixture array fails validateSpan(span) — e.g. a span missing a required property, wrong primitive type, bad enum value, or violating additionalProperties/required constraints defined in the span JSON schema.

Common situations: Hand-editing a spans fixture JSON and dropping or renaming a required field; a code change altered the span shape (new required field, renamed key) without updating fixtures; a generator wrote a span with a numeric value where a string id was expected.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• ${fixtureFiles[index]}…
• report fixture : theme transition disagrees with its…
• cave_agent_definition_invalid
• cave_context_ir_invalid
• cave_output_schema_mismatch

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/96287c897b7de2d5. Report an issue: GitHub.

Appendix: source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:336

    const perturbed = dataset.cases.some((entry) => entry.perturbation !== "none");
    if (perturbed !== item.eval_pack.graders.includes("guard_respected")) {
      throw new Error(`${where} guard_respected grader ${perturbed ? "missing for" : "declared without"} perturbed dataset cases`);
    }
    for (const proof of item.replay.trial_proofs) {
      if (!datasetCaseIDs.has(proof.dataset_case_id)) throw new Error(`${where} replay trial ${proof.id} replays a case outside the composed dataset`);
      for (const arm of [proof.baseline, proof.candidate]) {
        const graders = arm.grader_results.map((grader) => grader.grader).sort();
        const required = [...item.eval_pack.graders].sort();
        if (graders.join(",") !== required.join(",")) {
          throw new Error(`${where} replay trial ${proof.id} graded ${graders.join(",")}, want exactly ${required.join(",")}`);
        }
      }
    }
  }
}
if (!Array.isArray(spans) || spans.length === 0) throw new Error("canonical span fixture is empty");
for (const [index, span] of spans.entries()) {
  if (!validateSpan(span)) throw new Error(`canonical span ${index}: ${ajv.errorsText(validateSpan.errors)}`);
}
if (relationshipCount === 0) throw new Error("no theme relationship reached the shared-analysis-unit threshold in any fixture");
if (motifCount === 0) throw new Error("no workflow motif recurred across two variants of one task family in any fixture");
console.log(`validated ${reports.length} generated improvement report(s), ${relationshipCount} theme relationship(s), ${motifCount} workflow motif(s) and ${spans.length} canonical spans`);

View on GitHub (pinned to 3ee70a1026)