{"record":{"id":"e1c079252c4e1e14","repo":"santifer/career-ops","slug":"missing-score-summary-block-from-model-output","errorCode":null,"errorMessage":"Missing SCORE_SUMMARY block from model output","messagePattern":"Missing SCORE_SUMMARY block from model output","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"batch-evaluate-gemini.mjs","lineNumber":253,"sourceCode":"\n  console.log(`\\n========================================`);\n  console.log(`🔄 Processing [${idx}]: ${companyHint} - ${titleHint}`);\n  console.log(`🔗 URL: ${url}`);\n\n  try {\n    const jdText = await scrapeUrl(browser, url);\n    if (!jdText || jdText.length < 100) {\n      throw new Error('Extracted text too short (likely blocked or empty)');\n    }\n\n    console.log(`🧠 Calling Gemini (${modelName})...`);\n    const evaluationText = await _evaluate(`URL: ${url}\\n\\n${jdText}`);\n\n    // Parse output\n    const summaryMatch = evaluationText.match(/---SCORE_SUMMARY---\\s*([\\s\\S]*?)---END_SUMMARY---/);\n    if (!summaryMatch) {\n      console.error('Missing SCORE_SUMMARY block from model output:\\n' + evaluationText.slice(0, 500));\n      throw new Error('Missing SCORE_SUMMARY block from model output');\n    }\n    \n    const block = summaryMatch[1];\n    const extract = (key) => {\n      const m = block.match(new RegExp(`^\\\\s*${key}:\\\\s*(.+)$`, 'mi'));\n      return m ? m[1].trim() : 'unknown';\n    };\n\n    const company = extract('COMPANY');\n    const role = extract('ROLE');\n    const score = extract('SCORE');\n    const archetype = extract('ARCHETYPE');\n    const legitimacy = extract('LEGITIMACY');\n\n    // Save\n    mkdirSync(PATHS.reports, { recursive: true });\n    mkdirSync(PATHS.trackerAdditions, { recursive: true });\n","sourceCodeStart":235,"sourceCodeEnd":271,"githubUrl":"https://github.com/santifer/career-ops/blob/60398d6549a46f5266929538af21cfab94badc75/batch-evaluate-gemini.mjs#L235-L271","documentation":"The Gemini call succeeded and returned evaluationText, but it does not contain the '---SCORE_SUMMARY--- ... ---END_SUMMARY---' delimited block the parser regex (/---SCORE_SUMMARY---\\s*([\\s\\S]*?)---END_SUMMARY---/) requires. The model drifted from the output contract: it ignored the delimiters, got truncated before closing the block (output token limit), declined the content, or a safety stop fired. The script logs the first 500 characters of the raw model output before throwing, which is the primary diagnostic.","triggerScenarios":"A smaller/cheaper model tier ignoring delimiter formatting; output truncated by maxOutputTokens so ---END_SUMMARY--- never appears; the model refusing content it flags (personal data, suspicious JD text); the evaluation prompt edited so prompt delimiters and the parser regex no longer match; API safety settings blocking completion.","commonSituations":"Lowering spend_tier/model in config for cost and hitting weaker format adherence; very long JDs pushing the summary past the output limit; after hand-editing the evaluation prompt in modes/ the delimiter contract broke.","solutions":["Inspect the logged raw output slice: a refusal/safety message means change the input; text cut mid-sentence means truncation.","Re-run the single entry: transient format drift often passes on retry.","If truncated, shorten the JD text sent to the model or raise the output token limit.","Pin a more instruction-following model (raise spend_tier) for batch runs.","If you edited the prompt, restore the exact ---SCORE_SUMMARY--- / ---END_SUMMARY--- delimiters so prompt and parser regex match again."],"exampleFix":"// before (prompt ends loosely)\n...then give your verdict.\n\n// after (pin the contract in the prompt)\n...then finish your response with exactly:\n---SCORE_SUMMARY---\nCOMPANY: ...\nROLE: ...\nSCORE: ...\nARCHETYPE: ...\n---END_SUMMARY---","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"let evaluationText;\nfor (let attempt = 0; attempt < 2; attempt++) {\n  evaluationText = await _evaluate(jd);\n  if (/---SCORE_SUMMARY---[\\s\\S]*?---END_SUMMARY---/.test(evaluationText)) break;\n  if (attempt === 1) {\n    fs.writeFileSync(`batch/raw-output-${Date.now()}.txt`, evaluationText); // quarantine for manual parse\n    throw new Error('Missing SCORE_SUMMARY block from model output');\n  }\n}","preventionTips":["Keep the delimiter contract in one place: prompt and parser regex must use identical ---SCORE_SUMMARY---/---END_SUMMARY--- strings.","Use a strongly instruction-following model for batch runs; save the cheap tier for triage.","Cap JD input length so long postings cannot push the summary past the output limit."],"tags":["llm-output","gemini","parsing","structured-output"],"backgroundTag":"llm-output-parsing-failed","analyzedSha":"60398d6549a46f5266929538af21cfab94badc75","analyzedAt":"2026-08-20T23:00:06.764Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}