{"record":{"id":"84d11012ca69aa45","repo":"ruvnet/ruflo","slug":"frozen-human-eval-hash-mismatch-set-has-drifted","errorCode":null,"errorMessage":"frozen human eval hash mismatch — set has drifted (got ${corpusHash}, pinned ${FROZEN_HUMAN_EVAL_HASH}); supersede with a new versioned file, do not edit","messagePattern":"frozen human eval hash mismatch — set has drifted \\(got (.+?), pinned (.+?)\\); supersede with a new versioned file, do not edit","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"critical","filePath":"v3/@claude-flow/cli/src/services/harness-frozen-eval.ts","lineNumber":66,"sourceCode":"  } catch { /* not resolvable in this context */ }\n  candidates.push(path.resolve(__dirname, '..', '..', '..', FROZEN_HUMAN_EVAL_FILE)); // dist/src/services → pkg root\n  candidates.push(path.resolve(__dirname, '..', '..', FROZEN_HUMAN_EVAL_FILE));        // src/services → pkg root\n  for (const c of candidates) if (fs.existsSync(c)) return c;\n  return null;\n}\n\n/**\n * Load + verify the frozen human eval set. Throws if missing or if its content\n * hash != the pinned FROZEN_HUMAN_EVAL_HASH (the \"frozen\" guarantee).\n */\nexport function loadFrozenHumanEval(): FrozenHumanEval {\n  const p = locate();\n  if (!p) throw new Error(`frozen human eval set not found (${FROZEN_HUMAN_EVAL_FILE})`);\n  const parsed = JSON.parse(fs.readFileSync(p, 'utf-8')) as { version?: string; tasks?: HumanEvalTask[] };\n  const tasks = parsed.tasks ?? [];\n  const corpusHash = humanEvalHash(tasks);\n  if (corpusHash !== FROZEN_HUMAN_EVAL_HASH) {\n    throw new Error(`frozen human eval hash mismatch — set has drifted (got ${corpusHash}, pinned ${FROZEN_HUMAN_EVAL_HASH}); supersede with a new versioned file, do not edit`);\n  }\n  return { version: parsed.version ?? FROZEN_HUMAN_EVAL_VERSION, tasks, corpusHash };\n}\n","sourceCodeStart":48,"sourceCodeEnd":70,"githubUrl":"https://github.com/ruvnet/ruflo/blob/6b01dc5a687b26b3e218f796de45ec51f8fa9e8c/v3/@claude-flow/cli/src/services/harness-frozen-eval.ts#L48-L70","documentation":"Thrown by loadFrozenHumanEval() when the frozen eval file's content hash does not equal the pinned constant FROZEN_HUMAN_EVAL_HASH. This is the tamper-evidence guarantee: the set cannot silently drift. ANY edit (even reordering keys, whitespace that survives canonicalization, or swapping a task) trips it. The message explicitly says to supersede with a new versioned file, not to edit in place.","triggerScenarios":"The file .claude/eval/human-relevance-frozen-v1.json was edited, reformatted, partially overwritten, or replaced with a different version. The hash is computed over canon-sorted tasks, so structural or content changes both trigger it.","commonSituations":"A developer hand-edited the eval file to add a task; a formatter/linter rewrote the JSON; a package downgrade/upgrade shipped a different eval version; a merge conflict resolution altered content; an actual tampering attempt.","solutions":["DO NOT edit the file — restore it from git or reinstall the exact package version.","To change the eval set legitimately, create a NEW versioned file (e.g. human-relevance-frozen-v2.json) and update FROZEN_HUMAN_EVAL_HASH + FROZEN_HUMAN_EVAL_VERSION constants in source.","Verify no build step (prettier, eslint --fix) is rewriting the JSON.","Confirm package integrity: npm pack --dry-run or compare against the published tarball."],"exampleFix":"# restore the canonical file\ngit checkout -- .claude/eval/human-relevance-frozen-v1.json\n# OR supersede properly: create v2 and update the pinned constants in harness-frozen-eval.ts","handlingStrategy":"try-catch","validationCode":"import { humanEvalHash, FROZEN_HUMAN_EVAL_HASH } from './harness-frozen-eval.js';\nfunction evalFileIntact(path: string): boolean {\n  try {\n    const { tasks } = JSON.parse(fs.readFileSync(path, 'utf8'));\n    return humanEvalHash(tasks) === FROZEN_HUMAN_EVAL_HASH;\n  } catch { return false; }\n}","typeGuard":null,"tryCatchPattern":"try {\n  loadFrozenHumanEval();\n} catch (e) {\n  if (e instanceof Error && /hash mismatch/.test(e.message)) {\n    throw new Error('FATAL: frozen eval tamper detected — restore from git or supersede with a new versioned file');\n  }\n  throw e;\n}","preventionTips":["Never edit the frozen eval file — supersede with a new versioned file and update the pinned hash constant.","Add a CI check that verifies the eval file hash matches the pinned constant after build.","Ensure no formatter/linter rewrites the JSON.","Verify package integrity on install (npm pack --dry-run)."],"tags":["security","integrity","eval","tamper-detection","anchor"],"backgroundTag":null,"analyzedSha":"6b01dc5a687b26b3e218f796de45ec51f8fa9e8c","analyzedAt":"2026-08-12T13:20:50.148Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}