{"record":{"id":"2b809f2a1653254f","repo":"Yeachan-Heo/oh-my-codex","slug":"sandbox-md-frontmatter-evaluator-format-must-be-js","errorCode":null,"errorMessage":"sandbox.md frontmatter evaluator.format must be json in autoresearch v1.","messagePattern":"sandbox\\.md frontmatter evaluator\\.format must be json in autoresearch v1\\.","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"src/autoresearch/contracts.ts","lineNumber":172,"sourceCode":"  }\n\n  const evaluator = evaluatorRaw as { command?: unknown; format?: unknown; keep_policy?: unknown };\n  const command = typeof evaluator.command === 'string'\n    ? evaluator.command.trim()\n    : '';\n  const format = typeof evaluator.format === 'string'\n    ? evaluator.format.trim().toLowerCase()\n    : '';\n  const keepPolicy = parseKeepPolicy(evaluator.keep_policy);\n\n  if (!command) {\n    throw contractError(EVALUATOR_COMMAND_ERROR);\n  }\n  if (!format) {\n    throw contractError(EVALUATOR_FORMAT_REQUIRED_ERROR);\n  }\n  if (format !== 'json') {\n    throw contractError(EVALUATOR_FORMAT_JSON_ERROR);\n  }\n\n  return {\n    frontmatter: parsedFrontmatter,\n    evaluator: {\n      command,\n      format: 'json',\n      ...(keepPolicy ? { keep_policy: keepPolicy } : {}),\n    },\n    body,\n  };\n}\n\nexport function parseEvaluatorResult(raw: string): AutoresearchEvaluatorResult {\n  let parsed: unknown;\n  try {\n    parsed = JSON.parse(raw);\n  } catch {","sourceCodeStart":154,"sourceCodeEnd":190,"githubUrl":"https://github.com/Yeachan-Heo/oh-my-codex/blob/3ad79a8a6fe6e95fdbb8c00e40716fffe4011ce2/src/autoresearch/contracts.ts#L154-L190","documentation":"parseSandboxContract throws EVALUATOR_FORMAT_JSON_ERROR when evaluator.format is present but its lowercased/trimmed value is not exactly 'json'. Autoresearch v1 only supports JSON evaluator output, so any other format (text, table, csv, json5) is rejected.","triggerScenarios":"Frontmatter with 'format: text', 'format: csv', 'format: JSON5', or a typo like 'jason' after normalization.","commonSituations":"Porting a sandbox from another tool that emitted human-readable text; typos; attempting structured-but-not-JSON formats.","solutions":["Change the evaluator to emit JSON and set 'format: json'.","If your evaluator prints text, wrap/modify it to print a JSON object with boolean pass and optional numeric score, then declare json."],"exampleFix":"# before\nevaluator:\n  command: ./evaluate.sh --pretty\n  format: text\n\n# after\nevaluator:\n  command: ./evaluate.sh --json\n  format: json","handlingStrategy":"validation","validationCode":"function evaluatorFormatIsJson(content: string): boolean {\n  const m = /^[ \\t]+format:\\s*(\\S+)/m.exec(content);\n  return !!m && m[1].trim().toLowerCase() === 'json';\n}","typeGuard":"const isJsonFormat = (ev: { format?: unknown }): boolean =>\n  typeof ev.format === 'string' && ev.format.trim().toLowerCase() === 'json';","tryCatchPattern":"try {\n  parseSandboxContract(content);\n} catch (err) {\n  if ((err as Error).message.includes('format must be json')) {\n    // switch evaluator output to a JSON object and set format: json\n  }\n  throw err;\n}","preventionTips":["Only 'json' is supported in v1 — make evaluators print a JSON object.","Update evaluator scripts before changing sandbox.md to json format."],"tags":["yaml","frontmatter","evaluator","format-validation"],"backgroundTag":"unsupported-config-value","analyzedSha":"3ad79a8a6fe6e95fdbb8c00e40716fffe4011ce2","analyzedAt":"2026-08-27T22:18:39.783Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}