{"record":{"id":"71300d0c9acb528c","repo":"Yeachan-Heo/oh-my-codex","slug":"completion-requires-real-validation-evidence-not","errorCode":null,"errorMessage":"Completion requires real validation evidence, not placeholder evaluator text.","messagePattern":"Completion requires real validation evidence, not placeholder evaluator text\\.","errorType":"validation","errorClass":"GoalWorkflowValidationError","httpStatus":null,"severity":"error","filePath":"src/goal-workflows/validation.ts","lineNumber":44,"sourceCode":"    : input.status === 'blocker'\n      ? 'blocked'\n      : 'failed';\n  return {\n    status,\n    summary: input.summary.trim(),\n    artifactPath: input.artifactPath?.trim() || undefined,\n    checkedAt: iso(input.checkedAt),\n  };\n}\n\nexport function assertGoalWorkflowCanComplete(validation: GoalWorkflowValidationSummary | undefined): void {\n  if (!validation) throw new GoalWorkflowValidationError('Completion requires a validation artifact.');\n  if (validation.status !== 'validation_passed') {\n    throw new GoalWorkflowValidationError(`Completion requires validation_passed; got ${validation.status}.`);\n  }\n  if (!validation.artifactPath?.trim()) throw new GoalWorkflowValidationError('Completion requires a validation artifact path.');\n  if (hasPlaceholderEvidence(validation.summary)) {\n    throw new GoalWorkflowValidationError('Completion requires real validation evidence, not placeholder evaluator text.');\n  }\n}\n","sourceCodeStart":26,"sourceCodeEnd":47,"githubUrl":"https://github.com/Yeachan-Heo/oh-my-codex/blob/3ad79a8a6fe6e95fdbb8c00e40716fffe4011ce2/src/goal-workflows/validation.ts#L26-L47","documentation":"Thrown by assertGoalWorkflowCanComplete when a goal workflow run attempts to transition to completed but the validation artifact's summary contains placeholder evaluator text (detected via hasPlaceholderEvidence). The library refuses to complete runs whose validation evidence looks like a stub or template rather than genuine evaluator output, enforcing real validation before completion.","triggerScenarios":"Calling transitionGoalWorkflowRun to complete a run where validation.status === 'validation_passed' and artifactPath is set, but validation.summary matches placeholder patterns (e.g. 'TODO', 'placeholder', 'lorem ipsum', template text emitted by a mock evaluator).","commonSituations":"Using a stub/mock evaluator during integration tests and forgetting to substitute a realistic summary; CI pipelines that auto-complete runs with templated validation summaries; migrating from an older version that didn't check for placeholder evidence.","solutions":["Inspect validation.summary for the run and replace placeholder text with actual evaluator output (metrics, findings, pass criteria), then retry the transition","If running tests, configure the test evaluator to produce a non-placeholder summary instead of a hardcoded stub","Check hasPlaceholderEvidence's patterns to see exactly which strings are rejected and ensure your summary avoids them","If the validation was genuinely performed, regenerate the artifact with a detailed real summary and re-attach it to the run"],"exampleFix":"// before\nawait transitionGoalWorkflowRun(runId, 'completed'); // validation.summary = 'TODO: fill in results'\n\n// after\nvalidation.summary = 'Evaluator ran 42 checks: 42 passed, 0 failed. Coverage 91%. No regressions detected.';\nawait transitionGoalWorkflowRun(runId, 'completed');","handlingStrategy":"validation","validationCode":"import { hasPlaceholderEvidence } from './src/goal-workflows/validation.js';\n\nfunction canComplete(validation) {\n  return validation?.status === 'validation_passed'\n    && !!validation.artifactPath?.trim()\n    && !hasPlaceholderEvidence(validation.summary ?? '');\n}","typeGuard":"interface ValidationArtifact { status: string; artifactPath?: string; summary?: string }\nfunction isCompletableValidation(v: ValidationArtifact | undefined | null): v is ValidationArtifact {\n  return !!v && v.status === 'validation_passed' && !!v.artifactPath?.trim()\n    && !hasPlaceholderEvidence(v.summary ?? '');\n}","tryCatchPattern":"try {\n  await transitionGoalWorkflowRun(runId, 'completed');\n} catch (err) {\n  if (err instanceof GoalWorkflowValidationError && /placeholder/.test(err.message)) {\n    // regenerate real validation evidence, then retry\n  } else throw err;\n}","preventionTips":["Never complete workflows with mock/templated evaluator summaries","Run hasPlaceholderEvidence on generated summaries in your evaluator's test suite","Treat 'validation_passed' + placeholder summary as a CI failure in your own pipelines"],"tags":["goal-workflow","validation","state-machine","placeholder"],"backgroundTag":"workflow-validation-failed","analyzedSha":"3ad79a8a6fe6e95fdbb8c00e40716fffe4011ce2","analyzedAt":"2026-08-27T22:18:39.783Z","schemaVersion":2},"datasetVersion":"2026-08-28T00:17:15.603Z"}