Yeachan-Heo/oh-my-codex · error · Error

Evaluator command is still a placeholder/template. Refine fu

Error message

Evaluator command is still a placeholder/template. Refine further before launch.

What it means

Before launching, ensureLaunchReadyEvaluator() verifies the evaluator command via isLaunchReadyEvaluatorCommand(); if the command is still a placeholder/template it throws, blocking launch until the user refines it. This guards against launching autoresearch with an unconfigured evaluator.

Source

Thrown at src/cli/autoresearch-guided.ts:207

function shouldFallbackFromStructuredQuestion(error: unknown): boolean {
	if (error instanceof OmxQuestionError) {
		if (
			error.code === "worker_blocked"
			|| error.code === "team_blocked"
			|| error.code === "active_execution_mode_blocked"
		) {
			return false;
		}
		return true;
	}

	const message = error instanceof Error ? error.message : String(error);
	return /omx question/i.test(message);
}

function ensureLaunchReadyEvaluator(command: string): void {
	if (!isLaunchReadyEvaluatorCommand(command)) {
		throw new Error(
			"Evaluator command is still a placeholder/template. Refine further before launch.",
		);
	}
}

export function buildAutoresearchDeepInterviewPrompt(
	seedInputs: AutoresearchSeedInputs = {},
): string {
	const seedLines = [
		`- topic: ${seedInputs.topic?.trim() || "(none)"}`,
		`- evaluator: ${seedInputs.evaluatorCommand?.trim() || "(none)"}`,
		`- keep_policy: ${seedInputs.keepPolicy || "(none)"}`,
		`- slug: ${seedInputs.slug?.trim() || "(none)"}`,
	];

	return [
		"$deep-interview --autoresearch",
		"Run the deep-interview skill in autoresearch mode for `$autoresearch`.",

View on GitHub (pinned to 3ad79a8a6f)

Solutions

Choose 'refine further' at the next-step prompt and edit the evaluator command to a real command
Manually set a concrete evaluator command (shell command/script path) instead of the placeholder
Re-run the guided setup and complete the refinement stage before launch

Example fix

# before
evaluator: 'omx question "<EVALUATOR_PLACEHOLDER>"'
# after
evaluator: './scripts/evaluate.sh --rubric rubric.json'

Defensive patterns

Strategy: validation

Validate before calling

if (isPlaceholderEvaluator(evaluatorCommand)) {
  throw new SetupError('Replace the placeholder evaluator command before launch');
}
// or surface a refine prompt instead of launching

Type guard

const isLaunchReady = (cmd: string): boolean =>
  typeof cmd === 'string' && cmd.trim().length > 0 && !/placeholder|template|<.*>/i.test(cmd) && !/omx question/i.test(cmd);

Try / catch

try { materializeAutoresearchDeepInterviewResult(r); } catch (e) { if (e.message.includes('placeholder/template')) { sendBackToRefineStage(); } else throw e; }

Prevention

Always run at least one refine pass after the deep interview
Add CI checks that reject placeholder evaluator commands in configs

When it happens

Trigger: The deep-interview result's evaluator command still contains the template text (e.g. 'omx question ...' placeholder — note the isLaunchReadyError heuristic tests for /omx question/i) rather than a concrete evaluator command, and materializeAutoresearchDeepInterviewResult tries to finalize for launch.

Common situations: Skipping the refine loop in the guided interview so the default placeholder evaluator is never replaced, or automated flows that go straight from interview to launch.

Related errors

AI-assisted analysis of Yeachan-Heo/oh-my-codex@3ad79a8a6f (2026-08-27). Data as JSON: /api/errors/3d824b3ddf1b31a0. Report an issue: GitHub.