{"record":{"id":"7a5998333c31ca4f","repo":"paperclipai/paperclip","slug":"canonical-evalbook-generator-not-found-set-paperc","errorCode":null,"errorMessage":"Canonical Evalbook generator not found. Set PAPERCLIP_EVALBOOK_PROGRAM to ${EVAL_PROGRAM_RELATIVE_PATH} in a paperclip-evals checkout.","messagePattern":"Canonical Evalbook generator not found\\. Set PAPERCLIP_EVALBOOK_PROGRAM to (.+?) in a paperclip-evals checkout\\.","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"packages/paperclip-runner/scripts/render-runner-workflow-evalbook.mjs","lineNumber":283,"sourceCode":"    ? resolve(environment.PAPERCLIP_EVALS_ROOT, EVAL_PROGRAM_RELATIVE_PATH)\n    : null;\n  const program = await existingPath([\n    environment.PAPERCLIP_EVALBOOK_PROGRAM,\n    fromRoot,\n    resolve(packageRoot, \"../../.paperclip-evals\", EVAL_PROGRAM_RELATIVE_PATH),\n    resolve(\n      packageRoot,\n      \"../../../paperclip-evals\",\n      EVAL_PROGRAM_RELATIVE_PATH,\n    ),\n    resolve(\n      packageRoot,\n      \"../../../../paperclip-evals\",\n      EVAL_PROGRAM_RELATIVE_PATH,\n    ),\n  ]);\n  if (program !== null) return program;\n  throw new Error(\n    `Canonical Evalbook generator not found. Set PAPERCLIP_EVALBOOK_PROGRAM to ${EVAL_PROGRAM_RELATIVE_PATH} in a paperclip-evals checkout.`,\n  );\n}\n\nasync function run(command, args) {\n  await new Promise((accept, reject) => {\n    const child = spawn(command, args, { stdio: \"inherit\" });\n    child.once(\"error\", reject);\n    child.once(\"exit\", (code, signal) => {\n      if (code === 0) accept();\n      else\n        reject(\n          new Error(\n            `${command} exited ${code ?? `for signal ${signal ?? \"unknown\"}`}`,\n          ),\n        );\n    });\n  });","sourceCodeStart":265,"sourceCodeEnd":301,"githubUrl":"https://github.com/paperclipai/paperclip/blob/01ad8584922b5d85292b1723cae71fa0d9b07a19/packages/paperclip-runner/scripts/render-runner-workflow-evalbook.mjs#L265-L301","documentation":"resolveCanonicalEvalProgram searches known locations (including a sibling paperclip-evals checkout) for the canonical Evalbook generator script and throws when none exists. The error means the environment does not have the paperclip-evals repo checked out next to the package and PAPERCLIP_EVALBOOK_PROGRAM was not set to override the location.","triggerScenarios":"Running render-runner-workflow-evalbook.mjs in a fresh clone or CI container that only contains the paperclip monorepo; PAPERCLIP_EVALBOOK_PROGRAM unset and no ../../../../paperclip-evals directory present.","commonSituations":"Contributors cloning only the main repo; CI runners missing the sibling checkout step; moving the repo so the relative ../../.. path no longer resolves to the evals checkout.","solutions":["Clone the paperclip-evals repository as a sibling checkout at the expected relative path","Set PAPERCLIP_EVALBOOK_PROGRAM to the absolute path of the generator (EVAL_PROGRAM_RELATIVE_PATH inside a paperclip-evals checkout)","In CI, add a checkout step for paperclip-evals into the expected adjacent directory","Verify the resolved path with ls and adjust PAPERCLIP_EVALBOOK_PROGRAM accordingly"],"exampleFix":"// before\nnode render-runner-workflow-evalbook.mjs ...\n// after\nPAPERCLIP_EVALBOOK_PROGRAM=/srv/checkouts/paperclip-evals/evals/paperclip-runner/tools/eval_program.py \\\n  node render-runner-workflow-evalbook.mjs ...","handlingStrategy":"fallback","validationCode":"const program = process.env.PAPERCLIP_EVALBOOK_PROGRAM; if (!program || !existsSync(program)) throw new Error('Set PAPERCLIP_EVALBOOK_PROGRAM to the eval generator path');","typeGuard":"null","tryCatchPattern":"try { await renderRunnerWorkflowEvalbook(...); } catch (e) { if (e.message.includes('Canonical Evalbook generator not found')) console.error('Clone paperclip-evals or set PAPERCLIP_EVALBOOK_PROGRAM'); throw e; }","preventionTips":["Set PAPERCLIP_EVALBOOK_PROGRAM in CI secrets/env config","Document the sibling-checkout requirement in setup docs","Check for the eval program path early in setup scripts"],"tags":["missing-dependency","environment","evalbook"],"backgroundTag":"missing-env-var","analyzedSha":"01ad8584922b5d85292b1723cae71fa0d9b07a19","analyzedAt":"2026-09-10T03:14:50.855Z","contentChangedAt":"2026-09-10T03:14:50.855Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}