santifer/career-ops · error · Error

Gemini returned an invalid career-ops report: ${issues.join(

Error message

Gemini returned an invalid career-ops report: ${issues.join('; ')}

What it means

validateEvaluationShape checks that Gemini's raw text output matches the career-ops report contract: headers for Blocks A through G must be present, a SCORE_SUMMARY block delimited by ---SCORE_SUMMARY--- / ---END_SUMMARY--- must exist, it must contain COMPANY, ROLE, ARCHETYPE, and LEGITIMACY fields (ROLE/ARCHETYPE/LEGITIMACY cannot be 'unknown'), and SCORE must be a finite number in 0–5. All failures are collected and reported in one message.

Source

Thrown at gemini-eval.mjs:209

  } else {
    const summaryBlock = summary[1];
    for (const key of ['COMPANY', 'ROLE', 'ARCHETYPE', 'LEGITIMACY']) {
      const field = summaryBlock.match(new RegExp(`^\\s*${key}:\\s*(.+)$`, 'mi'));
      const value = field?.[1]?.trim() ?? '';
      if (!value || (key !== 'COMPANY' && value.toLowerCase() === 'unknown')) {
        issues.push(`SCORE_SUMMARY ${key} is required`);
      }
    }

    const score = summaryBlock.match(/^\s*SCORE:\s*([0-9]+(?:\.[0-9]+)?)/mi);
    const scoreValue = score ? Number(score[1]) : NaN;
    if (!Number.isFinite(scoreValue) || scoreValue < 0 || scoreValue > 5) {
      issues.push('SCORE_SUMMARY score must be a number between 0 and 5');
    }
  }

  if (issues.length > 0) {
    throw new Error(`Gemini returned an invalid career-ops report: ${issues.join('; ')}`);
  }
}

function slugifyCompany(value) {
  return String(value || '')
    .toLowerCase()
    .replace(/[^a-z0-9]+/g, '-')
    .replace(/^-|-$/g, '') || 'unknown';
}

function tsvSafe(value) {
  return String(value ?? '').replace(/[\t\r\n]+/g, ' ').trim();
}

function normalizedTrackerScore(value) {
  const clean = tsvSafe(value);
  if (!clean || clean === '?') return 'N/A';
  return /\/5$/i.test(clean) ? clean : `${clean}/5`;

View on GitHub (pinned to 9b17a8ac97)

Solutions

Re-run the evaluation — model output can vary between calls.
Inspect the raw Gemini output to see which block or field is missing or malformed.
If output is truncated, reduce the input size (shorten the JD or cv.md context) or increase the model output token limit.
Ensure the prompt template explicitly requests all Blocks A–G and the SCORE_SUMMARY block.

Defensive patterns

Strategy: try-catch

Try / catch

// validateEvaluationShape is called internally; callers wrap the Gemini call
try {
  const reportText = await callGemini(prompt);
  validateEvaluationShape(reportText);
  // proceed to write report
} catch (err) {
  if (err.message.includes('invalid career-ops report')) {
    console.error('Gemini output failed validation:', err.message);
    // Retry, or log raw output for inspection
    process.exit(1);
  }
  throw err;
}

Prevention

Re-run evaluations on validation failure — model output varies.
Keep prompts under the model's output token limit to avoid truncation.
Ensure the prompt template explicitly requests Blocks A–G and the SCORE_SUMMARY block.
Log raw model output alongside parsed reports for debugging.

When it happens

Trigger: Gemini truncated output mid-report (missing later blocks); omitted SCORE_SUMMARY delimiters; wrote 'unknown' for ROLE or ARCHETYPE; SCORE was outside 0-5 or non-numeric; model restructured the output format; token limit cut off the response.

Common situations: Model output instability across runs; the prompt template was changed and no longer requests a required block; a very long JD pushed the response over the output token limit; model returned a refusal or error instead of the report.

Related errors

AI-assisted analysis of santifer/career-ops@9b17a8ac97 (2026-08-13). Data as JSON: /api/errors/6489361f1022f4d9. Report an issue: GitHub.