{"record":{"id":"3d824b3ddf1b31a0","repo":"Yeachan-Heo/oh-my-codex","slug":"evaluator-command-is-still-a-placeholder-template","errorCode":null,"errorMessage":"Evaluator command is still a placeholder/template. Refine further before launch.","messagePattern":"Evaluator command is still a placeholder/template\\. Refine further before launch\\.","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"src/cli/autoresearch-guided.ts","lineNumber":207,"sourceCode":"function shouldFallbackFromStructuredQuestion(error: unknown): boolean {\n\tif (error instanceof OmxQuestionError) {\n\t\tif (\n\t\t\terror.code === \"worker_blocked\"\n\t\t\t|| error.code === \"team_blocked\"\n\t\t\t|| error.code === \"active_execution_mode_blocked\"\n\t\t) {\n\t\t\treturn false;\n\t\t}\n\t\treturn true;\n\t}\n\n\tconst message = error instanceof Error ? error.message : String(error);\n\treturn /omx question/i.test(message);\n}\n\nfunction ensureLaunchReadyEvaluator(command: string): void {\n\tif (!isLaunchReadyEvaluatorCommand(command)) {\n\t\tthrow new Error(\n\t\t\t\"Evaluator command is still a placeholder/template. Refine further before launch.\",\n\t\t);\n\t}\n}\n\nexport function buildAutoresearchDeepInterviewPrompt(\n\tseedInputs: AutoresearchSeedInputs = {},\n): string {\n\tconst seedLines = [\n\t\t`- topic: ${seedInputs.topic?.trim() || \"(none)\"}`,\n\t\t`- evaluator: ${seedInputs.evaluatorCommand?.trim() || \"(none)\"}`,\n\t\t`- keep_policy: ${seedInputs.keepPolicy || \"(none)\"}`,\n\t\t`- slug: ${seedInputs.slug?.trim() || \"(none)\"}`,\n\t];\n\n\treturn [\n\t\t\"$deep-interview --autoresearch\",\n\t\t\"Run the deep-interview skill in autoresearch mode for `$autoresearch`.\",","sourceCodeStart":189,"sourceCodeEnd":225,"githubUrl":"https://github.com/Yeachan-Heo/oh-my-codex/blob/3ad79a8a6fe6e95fdbb8c00e40716fffe4011ce2/src/cli/autoresearch-guided.ts#L189-L225","documentation":"Before launching, ensureLaunchReadyEvaluator() verifies the evaluator command via isLaunchReadyEvaluatorCommand(); if the command is still a placeholder/template it throws, blocking launch until the user refines it. This guards against launching autoresearch with an unconfigured evaluator.","triggerScenarios":"The deep-interview result's evaluator command still contains the template text (e.g. 'omx question ...' placeholder — note the isLaunchReadyError heuristic tests for /omx question/i) rather than a concrete evaluator command, and materializeAutoresearchDeepInterviewResult tries to finalize for launch.","commonSituations":"Skipping the refine loop in the guided interview so the default placeholder evaluator is never replaced, or automated flows that go straight from interview to launch.","solutions":["Choose 'refine further' at the next-step prompt and edit the evaluator command to a real command","Manually set a concrete evaluator command (shell command/script path) instead of the placeholder","Re-run the guided setup and complete the refinement stage before launch"],"exampleFix":"# before\nevaluator: 'omx question \"<EVALUATOR_PLACEHOLDER>\"'\n# after\nevaluator: './scripts/evaluate.sh --rubric rubric.json'","handlingStrategy":"validation","validationCode":"if (isPlaceholderEvaluator(evaluatorCommand)) {\n  throw new SetupError('Replace the placeholder evaluator command before launch');\n}\n// or surface a refine prompt instead of launching","typeGuard":"const isLaunchReady = (cmd: string): boolean =>\n  typeof cmd === 'string' && cmd.trim().length > 0 && !/placeholder|template|<.*>/i.test(cmd) && !/omx question/i.test(cmd);","tryCatchPattern":"try { materializeAutoresearchDeepInterviewResult(r); } catch (e) { if (e.message.includes('placeholder/template')) { sendBackToRefineStage(); } else throw e; }","preventionTips":["Always run at least one refine pass after the deep interview","Add CI checks that reject placeholder evaluator commands in configs"],"tags":["placeholder-value","launch-gate","autoresearch"],"backgroundTag":"unconfigured-required-setting","analyzedSha":"3ad79a8a6fe6e95fdbb8c00e40716fffe4011ce2","analyzedAt":"2026-08-27T22:18:39.783Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}