Yeachan-Heo/oh-my-codex · error · GoalWorkflowValidationError

Completion requires real validation evidence, not placeholde

Error message

Completion requires real validation evidence, not placeholder evaluator text.

What it means

Thrown by assertGoalWorkflowCanComplete when a goal workflow run attempts to transition to completed but the validation artifact's summary contains placeholder evaluator text (detected via hasPlaceholderEvidence). The library refuses to complete runs whose validation evidence looks like a stub or template rather than genuine evaluator output, enforcing real validation before completion.

Source

Thrown at src/goal-workflows/validation.ts:44

    : input.status === 'blocker'
      ? 'blocked'
      : 'failed';
  return {
    status,
    summary: input.summary.trim(),
    artifactPath: input.artifactPath?.trim() || undefined,
    checkedAt: iso(input.checkedAt),
  };
}

export function assertGoalWorkflowCanComplete(validation: GoalWorkflowValidationSummary | undefined): void {
  if (!validation) throw new GoalWorkflowValidationError('Completion requires a validation artifact.');
  if (validation.status !== 'validation_passed') {
    throw new GoalWorkflowValidationError(`Completion requires validation_passed; got ${validation.status}.`);
  }
  if (!validation.artifactPath?.trim()) throw new GoalWorkflowValidationError('Completion requires a validation artifact path.');
  if (hasPlaceholderEvidence(validation.summary)) {
    throw new GoalWorkflowValidationError('Completion requires real validation evidence, not placeholder evaluator text.');
  }
}

View on GitHub (pinned to 3ad79a8a6f)

Solutions

1. Inspect validation.summary for the run and replace placeholder text with actual evaluator output (metrics, findings, pass criteria), then retry the transition
2. If running tests, configure the test evaluator to produce a non-placeholder summary instead of a hardcoded stub
3. Check hasPlaceholderEvidence's patterns to see exactly which strings are rejected and ensure your summary avoids them
4. If the validation was genuinely performed, regenerate the artifact with a detailed real summary and re-attach it to the run

Example fix

// before
await transitionGoalWorkflowRun(runId, 'completed'); // validation.summary = 'TODO: fill in results'

// after
validation.summary = 'Evaluator ran 42 checks: 42 passed, 0 failed. Coverage 91%. No regressions detected.';
await transitionGoalWorkflowRun(runId, 'completed');

import { hasPlaceholderEvidence } from './src/goal-workflows/validation.js';

function canComplete(validation) {
  return validation?.status === 'validation_passed'
    && !!validation.artifactPath?.trim()
    && !hasPlaceholderEvidence(validation.summary ?? '');
}

interface ValidationArtifact { status: string; artifactPath?: string; summary?: string }
function isCompletableValidation(v: ValidationArtifact | undefined | null): v is ValidationArtifact {
  return !!v && v.status === 'validation_passed' && !!v.artifactPath?.trim()
    && !hasPlaceholderEvidence(v.summary ?? '');
}

try {
  await transitionGoalWorkflowRun(runId, 'completed');
} catch (err) {
  if (err instanceof GoalWorkflowValidationError && /placeholder/.test(err.message)) {
    // regenerate real validation evidence, then retry
  } else throw err;
}

When it happens

Trigger: Calling transitionGoalWorkflowRun to complete a run where validation.status === 'validation_passed' and artifactPath is set, but validation.summary matches placeholder patterns (e.g. 'TODO', 'placeholder', 'lorem ipsum', template text emitted by a mock evaluator).

Common situations: Using a stub/mock evaluator during integration tests and forgetting to substitute a realistic summary; CI pipelines that auto-complete runs with templated validation summaries; migrating from an older version that didn't check for placeholder evidence.

Related errors

• invalid auth slot path
• Autoresearch goal ${mission.slug} cannot complete until prof
• autoresearch candidate artifact notes must be a string array
• autoresearch candidate artifact created_at is required
• catalog_manifest_invalid:${field}

AI-assisted analysis of Yeachan-Heo/oh-my-codex@3ad79a8a6f (2026-08-27). Data as JSON: /api/errors/71300d0c9acb528c. Report an issue: GitHub.