{"record":{"id":"b6f7b75d43a07092","repo":"JuliusBrussee/caveman","slug":"agent-conformance-mismatch-key","errorCode":null,"errorMessage":"agent conformance mismatch: ${key}","messagePattern":"agent conformance mismatch: (.+?)","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/shared/contracts/scripts/validate-schemas.mjs","lineNumber":75,"sourceCode":"    );\n  }\n}\nconst [claude, pi] = fixtures;\nif (claude.harness !== \"claude\" || pi.harness !== \"pi\") {\n  throw new Error(\"agent conformance fixtures must cover claude and pi\");\n}\n\nfor (const key of [\n  \"normalized_context_digest\",\n  \"plan_sha256\",\n  \"ordered_transform_ids\",\n  \"provider_visible_digest\",\n  \"recovery_handles\",\n  \"accounting_method\",\n  \"failure_fallback\",\n]) {\n  if (JSON.stringify(claude[key]) !== JSON.stringify(pi[key])) {\n    throw new Error(`agent conformance mismatch: ${key}`);\n  }\n}\nif (claude.build_sha256 === pi.build_sha256) {\n  throw new Error(\"adapter-specific build_sha256 values must differ\");\n}\nif (claude.failure_fallback !== \"original\") {\n  throw new Error(\"unknown adapter failure must preserve original bytes\");\n}\n\nconst middleware = JSON.parse(await readFile(path.join(packageRoot, \"..\", \"..\", \"sdk\", \"parity\", \"middleware.fixtures.json\"), \"utf8\"));\nfor (const [field, schemaName] of Object.entries({ capabilities: \"capabilities\", request: \"optimize\", plan: \"plan\", page: \"page\" })) {\n  const validate = ajv.getSchema(`https://caveman.so/schemas/middleware-${schemaName}.schema.json`);\n  if (!validate?.(middleware[field])) throw new Error(`middleware ${field}: ${ajv.errorsText(validate?.errors)}`);\n}\n\nconsole.log(\n  `validated ${files.length} JSON schemas and ${fixtureFiles.length} static agent contract fixtures (not executable parity)`,\n);","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/packages/shared/contracts/scripts/validate-schemas.mjs#L57-L93","documentation":"After confirming one claude and one pi fixture, the validator compares seven shared contract fields (normalized_context_digest, plan_sha256, ordered_transform_ids, provider_visible_digest, recovery_handles, accounting_method, failure_fallback) between the two fixtures using JSON.stringify equality. If any of these differ, it throws `agent conformance mismatch: <key>`, proving the two adapter implementations no longer agree on the static transform contract. The fixture contract records the digests/plan both adapters must produce from the same normalized context.","triggerScenarios":"Running the contracts validation script when a fixture's digest, plan_sha256, transform id list, recovery handles, accounting method, or failure_fallback was updated for one harness but not the other; regenerating digests after a transform change in only one codebase; hand-editing ordered_transform_ids to add a new transform in claude-conformance.json only; changing accounting_method or failure_fallback semantics for one adapter.","commonSituations":"A transform pipeline change (e.g. adding caveman.foo.v1) where one team regenerated their fixture and the other didn't; a merge that took one fixture from a branch but not its counterpart; intentional divergence for one adapter that needs the field moved to an adapter-specific section first; stale fixtures after bumping adapter_version without recomputing digests.","solutions":["Read the error's <key> suffix to see exactly which field diverged, then diff that field between claude-conformance.json and pi-conformance.json.","Recompute the shared value (digest, plan hash, transform id list) for BOTH adapters after the underlying change and regenerate both fixtures together.","If one adapter intentionally diverges, reconcile the contract: either fix the adapter, or move the divergent value out of the shared seven-key list and update validate-schemas.mjs accordingly (requires coordinating the wire contract).","Ensure both fixture files were updated in the same commit/PR so they can't drift again."],"exampleFix":"// before (pi-conformance.json)\n\"ordered_transform_ids\": [\"caveman.cache-preserve.v1\", \"caveman.tool-search.v1\"]\n\n// after (match claude-conformance.json)\n\"ordered_transform_ids\": [\"caveman.cache-preserve.v1\", \"caveman.tool-search.v1\", \"caveman.ccr-results.v1\"]","handlingStrategy":"validation","validationCode":"const keys = [\"normalized_context_digest\",\"plan_sha256\",\"ordered_transform_ids\",\"provider_visible_digest\",\"recovery_handles\",\"accounting_method\",\"failure_fallback\"];\nconst [a, b] = fixtures;\nconst mismatches = keys.filter(k => JSON.stringify(a[k]) !== JSON.stringify(b[k]));\nif (mismatches.length) throw new Error(`fixtures diverge before commit: ${mismatches.join(\",\")}`);","typeGuard":"function sharesContract(a, b) {\n  return [\"normalized_context_digest\",\"plan_sha256\"].every(k => a[k] === b[k]);\n}","tryCatchPattern":"try {\n  await runValidationScript();\n} catch (err) {\n  const m = err.message.match(/agent conformance mismatch: (\\w+)/);\n  if (m) {\n    console.error(`Field ${m[1]} differs between fixtures; regenerate both adapters' fixtures from the current pipeline.`);\n    process.exit(1);\n  }\n  throw err;\n}","preventionTips":["Regenerate both adapter fixtures in the same commit whenever transforms, plans, or digests change.","Add a pre-commit hook that diffs the seven shared keys between the two fixture files.","Never update digests by hand; derive them from the pipeline so both adapters compute identical values."],"tags":["build","fixtures","contract-parity","digest-mismatch"],"backgroundTag":"checksum-mismatch","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}