Yeachan-Heo/oh-my-codex · error · Error

Evaluator output must be a JSON object.

Error message

Evaluator output must be a JSON object.

What it means

parseEvaluatorResult throws when the evaluator output parses as valid JSON but is not a plain object — e.g. an array, a number, a string, or null. The contract requires a top-level JSON object so that pass/score fields can be read.

Source

Thrown at src/autoresearch/contracts.ts:195

    evaluator: {
      command,
      format: 'json',
      ...(keepPolicy ? { keep_policy: keepPolicy } : {}),
    },
    body,
  };
}

export function parseEvaluatorResult(raw: string): AutoresearchEvaluatorResult {
  let parsed: unknown;
  try {
    parsed = JSON.parse(raw);
  } catch {
    throw contractError('Evaluator output must be valid JSON with required boolean pass and optional numeric score.');
  }

  if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) {
    throw contractError('Evaluator output must be a JSON object.');
  }

  const result = parsed as Record<string, unknown>;
  if (typeof result.pass !== 'boolean') {
    throw contractError('Evaluator output must include boolean pass.');
  }
  if (result.score !== undefined && typeof result.score !== 'number') {
    throw contractError('Evaluator output score must be numeric when provided.');
  }

  return {
    pass: result.pass,
    ...(result.score === undefined ? {} : { score: result.score }),
  };
}

export async function loadAutoresearchMissionContract(missionDirArg: string): Promise<AutoresearchMissionContract> {
  const missionDir = resolve(missionDirArg);

View on GitHub (pinned to 3ad79a8a6f)

Solutions

1. Print a JSON object: '{"pass": true}' minimum.
2. Wrap array results as an object field (e.g. {"pass": true, "results": [...]}) with a top-level boolean pass.
3. Ensure the final stdout line/string is the object itself.

Example fix

# before
echo '[{"pass":true}]'

# after
echo '{"pass":true}'

function evaluatorOutputIsJsonObject(raw: string): boolean {
  try {
    const v = JSON.parse(raw);
    return !!v && typeof v === 'object' && !Array.isArray(v);
  } catch { return false; }
}

const isJsonObject = (v: unknown): v is Record<string, unknown> =>
  !!v && typeof v === 'object' && !Array.isArray(v);

try {
  parseEvaluatorResult(stdout);
} catch (err) {
  if ((err as Error).message.includes('must be a JSON object')) {
    // wrap arrays/scalars into an object with a boolean pass field
  }
  throw err;
}

When it happens

Trigger: Evaluator prints '[true]', '42', '"pass"', or 'null' — valid JSON but not an object; the !parsed || typeof !== 'object' || Array.isArray guard rejects it.

Common situations: Evaluators that print a bare score number or a JSON array of results; printing 'null' on no-op runs.

Related errors

• autoresearch candidate artifact must be a JSON object
• Evaluator output must be valid JSON with required boolean pa
• Evaluator output must include boolean pass.
• Evaluator output score must be numeric when provided.
• Invalid autoresearch-goal mission at ${repoRelative(cwd, pat

AI-assisted analysis of Yeachan-Heo/oh-my-codex@3ad79a8a6f (2026-08-27). Data as JSON: /api/errors/3b173a4076733b6d. Report an issue: GitHub.