{"record":{"id":"8aaa5e6c9bb3b5f3","repo":"ruvnet/ruflo","slug":"frozen-human-eval-set-not-found-frozen-human-ev","errorCode":null,"errorMessage":"frozen human eval set not found (${FROZEN_HUMAN_EVAL_FILE})","messagePattern":"frozen human eval set not found \\((.+?)\\)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"v3/@claude-flow/cli/src/services/harness-frozen-eval.ts","lineNumber":61,"sourceCode":"function locate(): string | null {\n  const candidates: string[] = [];\n  try {\n    const req = createRequire(import.meta.url);\n    candidates.push(path.join(path.dirname(req.resolve('@claude-flow/cli/package.json')), FROZEN_HUMAN_EVAL_FILE));\n  } catch { /* not resolvable in this context */ }\n  candidates.push(path.resolve(__dirname, '..', '..', '..', FROZEN_HUMAN_EVAL_FILE)); // dist/src/services → pkg root\n  candidates.push(path.resolve(__dirname, '..', '..', FROZEN_HUMAN_EVAL_FILE));        // src/services → pkg root\n  for (const c of candidates) if (fs.existsSync(c)) return c;\n  return null;\n}\n\n/**\n * Load + verify the frozen human eval set. Throws if missing or if its content\n * hash != the pinned FROZEN_HUMAN_EVAL_HASH (the \"frozen\" guarantee).\n */\nexport function loadFrozenHumanEval(): FrozenHumanEval {\n  const p = locate();\n  if (!p) throw new Error(`frozen human eval set not found (${FROZEN_HUMAN_EVAL_FILE})`);\n  const parsed = JSON.parse(fs.readFileSync(p, 'utf-8')) as { version?: string; tasks?: HumanEvalTask[] };\n  const tasks = parsed.tasks ?? [];\n  const corpusHash = humanEvalHash(tasks);\n  if (corpusHash !== FROZEN_HUMAN_EVAL_HASH) {\n    throw new Error(`frozen human eval hash mismatch — set has drifted (got ${corpusHash}, pinned ${FROZEN_HUMAN_EVAL_HASH}); supersede with a new versioned file, do not edit`);\n  }\n  return { version: parsed.version ?? FROZEN_HUMAN_EVAL_VERSION, tasks, corpusHash };\n}\n","sourceCodeStart":43,"sourceCodeEnd":70,"githubUrl":"https://github.com/ruvnet/ruflo/blob/fa13ee4ad60ac2090b1480656eb233521790d640/v3/@claude-flow/cli/src/services/harness-frozen-eval.ts#L43-L70","documentation":"ADR-176 freezes a public human-labeled eval set shipped inside the package at `.claude/eval/human-relevance-frozen-v1.json`; it is the anti-overfitting anchor for the flywheel. `locate()` probes candidate roots (the resolved package root, `__dirname/../../..` for dist, `__dirname/../..` for src) and if none contains the file, `loadFrozenHumanEval` throws — evaluation cannot proceed without the frozen set.","triggerScenarios":"Running a code path that loads the frozen set (e.g. the built-in anchor selection for the ruflo repository itself, or human-relevance deltas in receipts) from an install where the JSON is absent: bundlers that inline dist without sibling assets, Docker images copying only compiled JS, `files` allowlists in package.json omitting `.claude/eval`, or a pruned node_modules.","commonSituations":"Single-file bundles (esbuild/webpack) that drop non-JS assets; minimal container images; downstream forks repackaging the CLI; broken mirrors or partial installs; running a relocated copy of `dist/` without the package root.","solutions":["Reinstall the package (`npm i @claude-flow/cli`) so `.claude/eval/human-relevance-frozen-v1.json` exists under the package root","If you bundle or repackage the CLI, copy the `.claude/eval/` directory next to the compiled output so the `__dirname`-relative probes find it","Verify the file ships: inspect `npm pack` output or the package.json `files` list","Make sure you execute the installed package (its resolved root contains the file), not a moved copy of dist"],"exampleFix":"# before: Docker image copies only compiled JS\nCOPY dist /app/dist\n\n# after: package assets included so locate() probes succeed\nCOPY package.json /app/\nCOPY .claude/eval/human-relevance-frozen-v1.json /app/.claude/eval/","handlingStrategy":"validation","validationCode":"import { existsSync } from 'node:fs';\nimport { createRequire } from 'node:module';\nimport { fileURLToPath } from 'node:url';\nimport { dirname, join, resolve } from 'node:path';\n\nconst __dirname = dirname(fileURLToPath(import.meta.url));\nconst REL = join('.claude', 'eval', 'human-relevance-frozen-v1.json');\n\n// Mirrors locate(): true only if some candidate exists.\nfunction frozenEvalAvailable(): boolean {\n  const candidates: string[] = [];\n  try {\n    const req = createRequire(import.meta.url);\n    candidates.push(join(dirname(req.resolve('@claude-flow/cli/package.json')), REL));\n  } catch { /* unresolvable */ }\n  candidates.push(resolve(__dirname, '..', '..', '..', REL));\n  candidates.push(resolve(__dirname, '..', '..', REL));\n  return candidates.some((c) => existsSync(c));\n}\n\nif (!frozenEvalAvailable()) throw new Error('frozen eval asset missing — reinstall the package or fix bundling');","typeGuard":null,"tryCatchPattern":"try {\n  const frozen = loadFrozenHumanEval();\n} catch (e) {\n  if (e instanceof Error && e.message.includes('frozen human eval set not found')) {\n    // packaging defect, not runtime state: fail with actionable guidance\n    throw new Error('Install incomplete: .claude/eval/human-relevance-frozen-v1.json is missing from the package. Reinstall @claude-flow/cli or copy the asset next to the build output.');\n  }\n  throw e;\n}","preventionTips":["If you bundle the CLI, copy the `.claude/eval/` assets next to the output","Verify the file ships via `npm pack` before publishing forks","Smoke-test the frozen-eval load path in your Docker image build","Run from the installed package root, not a relocated dist copy"],"tags":["missing-file","packaging","frozen-eval","harness"],"backgroundTag":"missing-resource-file","analyzedSha":"fa13ee4ad60ac2090b1480656eb233521790d640","analyzedAt":"2026-08-18T21:34:22.708Z","contentChangedAt":"2026-08-18T21:34:22.708Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}