JuliusBrussee/caveman · error · Error

report fixture

Error message

report fixture ${reportPaths[index]}: ${ajv.errorsText(validateReport.errors)}

What it means

Each report fixture is validated against the continuous-improvement-report JSON Schema with Ajv. If `validateReport(report)` fails, the script throws with the fixture path and `ajv.errorsText(...)` describing every schema violation. This ensures reports consumed by the continuous-improvement pipeline conform to the published contract.

Solutions

Read the `ajv.errorsText` portion of the message — it names the exact JSON path and violated keyword.
Open the report fixture and fix the fields the error path points at.
Validate locally against packages/shared/contracts/schemas/continuous-improvement-report.schema.json (or with an IDE JSON-schema binding) before rerunning.
If the fixture came from a generator, regenerate with the current tool version instead of hand-editing.

Example fix

// before (fixture)
{ "themes": [{ "id": "t1", "lineage": { "transition": "new" } }] }        // registry_id missing
// after
{ "themes": [{ "id": "t1", "registry_id": "", "lineage": { "transition": "new", "predecessor_ids": [] } }] }

Defensive patterns

Strategy: validation

Validate before calling

import Ajv from "ajv";
import reportSchema from "../schemas/continuous-improvement-report.schema.json" with { type: "json" };
const ajv = new Ajv({ allErrors: true });
const validate = ajv.compile(reportSchema);
const report = JSON.parse(await readFile(reportPath, "utf8"));
if (!validate(report)) console.error(`pre-check ${reportPath}:`, ajv.errorsText(validate.errors));

Try / catch

try {
  await validateReports([reportPath], spansPath);
} catch (e) {
  if (String(e.message).startsWith("report fixture")) {
    console.error(e.message); // includes ajv.errorsText with exact JSON paths
    process.exit(1);
  }
  throw e;
}

Prevention

Bind the schema in your editor so fixtures are validated as you edit them.
Regenerate fixtures with the current pipeline instead of hand-editing after schema changes.
Run the validator in CI on every fixture change.
Keep a schema-version field in reports to detect stale generators.

When it happens

Trigger: Running the validator with a report JSON missing required properties (e.g. `themes` array items lacking `id`, `registry_id`, or `lineage`), wrong types (string instead of enum for `lineage.transition`), additional properties when additionalProperties is false, or a non-object root.

Common situations: Hand-authored test fixtures drifting from the schema after a schema version bump; a report generated by an older generator before new required fields (like lineage) were introduced; typos in field names; JSON exported with different casing.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/7db5bf7c15a440f2. Report an issue: GitHub.

Appendix: source

Thrown at packages/shared/contracts/scripts/validate-continuous-improvement.mjs:34

  readFile(path.join(packageRoot, "schemas", "continuous-improvement-report.schema.json"), "utf8").then(JSON.parse),
  readFile(path.join(packageRoot, "schemas", "canonical-span.schema.json"), "utf8").then(JSON.parse),
  readFile(spansPath, "utf8").then(JSON.parse),
]);
const reports = await Promise.all(reportPaths.map((reportPath) => readFile(reportPath, "utf8").then(JSON.parse)));

const ajv = new Ajv2020({ allErrors: true, strict: true });
const validateReport = ajv.compile(reportSchema);
const validateSpan = ajv.compile(spanSchema);
const contiguouslyContains = (haystack, needle) => {
  for (let start = 0; start + needle.length <= haystack.length; start += 1) {
    if (needle.every((operation, offset) => haystack[start + offset] === operation)) return true;
  }
  return false;
};
let relationshipCount = 0;
let motifCount = 0;
for (const [index, report] of reports.entries()) {
  if (!validateReport(report)) throw new Error(`report fixture ${reportPaths[index]}: ${ajv.errorsText(validateReport.errors)}`);
  for (const theme of report.themes) {
    // A durable registry id and a predecessor are the same claim: a theme that
    // continues an earlier one must carry both, and a new theme neither.
    const isNew = theme.lineage.transition === "new";
    if (isNew !== (theme.registry_id === "")) {
      throw new Error(`report fixture ${reportPaths[index]}: theme ${theme.id} transition ${theme.lineage.transition} disagrees with registry_id "${theme.registry_id}"`);
    }
    if (isNew !== (theme.lineage.predecessor_ids.length === 0)) {
      throw new Error(`report fixture ${reportPaths[index]}: theme ${theme.id} transition ${theme.lineage.transition} disagrees with its predecessor list`);
    }
    if (!isNew && !theme.lineage.predecessor_ids.includes(theme.registry_id)) {
      throw new Error(`report fixture ${reportPaths[index]}: theme ${theme.id} inherited a registry id that is not one of its predecessors`);
    }
  }

  // An opportunity names the exact pair of workflow variants it was emitted
  // from, and a safety finding carries no borrowed dollar figure: copying the
  // efficiency finding's alternative metrics and expected value would let the

View on GitHub (pinned to 3ee70a1026)