{"record":{"id":"bfd1573e55bef3d3","repo":"Yeachan-Heo/oh-my-codex","slug":"sandbox-md-frontmatter-must-define-an-evaluator-bl","errorCode":null,"errorMessage":"sandbox.md frontmatter must define an evaluator block.","messagePattern":"sandbox\\.md frontmatter must define an evaluator block\\.","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"src/autoresearch/contracts.ts","lineNumber":153,"sourceCode":"function parseKeepPolicy(raw: unknown): AutoresearchKeepPolicy | undefined {\n  if (raw === undefined) return undefined;\n  if (typeof raw !== 'string') {\n    throw contractError('sandbox.md frontmatter evaluator.keep_policy must be a string when provided.');\n  }\n  const normalized = raw.trim().toLowerCase();\n  if (!normalized) return undefined;\n  if (normalized === 'pass_only') return 'pass_only';\n  if (normalized === 'score_improvement') return 'score_improvement';\n  throw contractError('sandbox.md frontmatter evaluator.keep_policy must be one of: score_improvement, pass_only.');\n}\n\nexport function parseSandboxContract(content: string): ParsedSandboxContract {\n  const { frontmatter, body } = extractFrontmatter(content);\n  const parsedFrontmatter = parseSimpleYamlFrontmatter(frontmatter);\n  const evaluatorRaw = parsedFrontmatter.evaluator;\n\n  if (!evaluatorRaw || typeof evaluatorRaw !== 'object' || Array.isArray(evaluatorRaw)) {\n    throw contractError(EVALUATOR_BLOCK_ERROR);\n  }\n\n  const evaluator = evaluatorRaw as { command?: unknown; format?: unknown; keep_policy?: unknown };\n  const command = typeof evaluator.command === 'string'\n    ? evaluator.command.trim()\n    : '';\n  const format = typeof evaluator.format === 'string'\n    ? evaluator.format.trim().toLowerCase()\n    : '';\n  const keepPolicy = parseKeepPolicy(evaluator.keep_policy);\n\n  if (!command) {\n    throw contractError(EVALUATOR_COMMAND_ERROR);\n  }\n  if (!format) {\n    throw contractError(EVALUATOR_FORMAT_REQUIRED_ERROR);\n  }\n  if (format !== 'json') {","sourceCodeStart":135,"sourceCodeEnd":171,"githubUrl":"https://github.com/Yeachan-Heo/oh-my-codex/blob/3ad79a8a6fe6e95fdbb8c00e40716fffe4011ce2/src/autoresearch/contracts.ts#L135-L171","documentation":"parseSandboxContract throws EVALUATOR_BLOCK_ERROR when the parsed frontmatter has no 'evaluator' key, or evaluator is not a plain object (it is a string/array). The evaluator block is the core of the autoresearch v1 sandbox contract and must be a mapping containing command and format.","triggerScenarios":"sandbox.md frontmatter lacking an 'evaluator:' section; 'evaluator: ./run.sh' (scalar); or evaluator defined as a list. Occurs wherever parseSandboxContract is used: loading contracts, writing draft/deep-interview artifacts, parsing draft artifacts, and reading persisted results.","commonSituations":"Forgetting the evaluator block when authoring sandbox.md; inlining the command at the top level instead of under evaluator; malformed nesting that makes the parser store a scalar.","solutions":["Add an 'evaluator:' section header with nested 'command:' and 'format: json' lines.","Ensure evaluator is a mapping, not an inline scalar or list.","Validate sandbox.md with parseSandboxContract in a test or preflight script before running the mission."],"exampleFix":"# before\ncommand: ./run.sh\nformat: json\n\n# after\nevaluator:\n  command: ./run.sh\n  format: json","handlingStrategy":"validation","validationCode":"function hasEvaluatorBlock(content: string): boolean {\n  const fm = content.match(/^---\\r?\\n([\\s\\S]*?)\\r?\\n---/);\n  if (!fm) return false;\n  return /^[A-Za-z0-9_-]+:\\s*$/m.test(fm[1]) && /(^|\\n)evaluator:\\s*(\\n|$)/.test(fm[1]);\n}","typeGuard":"const hasEvaluatorObject = (fm: Record<string, unknown>): boolean =>\n  !!fm.evaluator && typeof fm.evaluator === 'object' && !Array.isArray(fm.evaluator);","tryCatchPattern":"try {\n  parseSandboxContract(content);\n} catch (err) {\n  if ((err as Error).message.includes('must define an evaluator block')) {\n    // add an 'evaluator:' section with command and format: json\n  }\n  throw err;\n}","preventionTips":["Always start from a canonical sandbox.md template containing the evaluator block.","Validate sandbox.md via parseSandboxContract in a preflight test.","Keep evaluator as a mapping, not an inline scalar."],"tags":["yaml","frontmatter","evaluator","contract-validation"],"backgroundTag":"missing-required-config-section","analyzedSha":"3ad79a8a6fe6e95fdbb8c00e40716fffe4011ce2","analyzedAt":"2026-08-27T22:18:39.783Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}