{"record":{"id":"9775738e58ea746c","repo":"srbhr/Resume-Matcher","slug":"content-too-large-for-json-extraction-len-conten","errorCode":null,"errorMessage":"Content too large for JSON extraction: {len(content)} bytes","messagePattern":"Content too large for JSON extraction: (.+?) bytes","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"apps/backend/app/llm.py","lineNumber":1111,"sourceCode":"    stripped = re.sub(r\"<think>.*?</think>\", \"\", content, flags=re.DOTALL)\n    # Also handle unclosed <think> tag (model may still be \"thinking\" at end)\n    stripped = re.sub(r\"<think>.*\", \"\", stripped, flags=re.DOTALL)\n    return stripped.strip()\n\n\ndef _extract_json(content: str, _depth: int = 0) -> str:\n    \"\"\"Extract JSON from LLM response, handling various formats.\n\n    LLM-001: Improved to detect and reject likely truncated JSON.\n    LLM-007: Improved error messages for debugging.\n    JSON-010: Added recursion depth and size limits.\n    \"\"\"\n    # JSON-010: Safety limits\n    if _depth > MAX_JSON_EXTRACTION_RECURSION:\n        raise ValueError(\n            f\"JSON extraction exceeded max recursion depth: {_depth}\")\n    if len(content) > MAX_JSON_CONTENT_SIZE:\n        raise ValueError(\n            f\"Content too large for JSON extraction: {len(content)} bytes\")\n\n    original = content\n\n    # Strip thinking model tags (deepseek-r1, qwq, etc.)\n    if \"<think>\" in content:\n        content = _strip_thinking_tags(content)\n\n    # Remove markdown code blocks\n    if \"```json\" in content:\n        content = content.split(\"```json\")[1].split(\"```\")[0]\n    elif \"```\" in content:\n        parts = content.split(\"```\")\n        if len(parts) >= 2:\n            content = parts[1]\n            # Remove language identifier if present (e.g., \"json\\n{...\")\n            if content.startswith((\"json\", \"JSON\")):\n                content = content[4:]","sourceCodeStart":1093,"sourceCodeEnd":1129,"githubUrl":"https://github.com/srbhr/Resume-Matcher/blob/116f9cc3b00e1ac91734a6c2679bf41ea64a0edc/apps/backend/app/llm.py#L1093-L1129","documentation":"_extract_json enforces MAX_JSON_CONTENT_SIZE as a JSON-010 safety limit; if the LLM response text exceeds it (len(content) > limit), it refuses to parse and raises this ValueError to bound memory/CPU on huge responses.","triggerScenarios":"complete_json gets a response whose content length exceeds MAX_JSON_CONTENT_SIZE bytes — e.g. the model echoed the whole resume plus verbose enhancements, or generated runaway repetitive output.","commonSituations":"Very large resumes/jobs fed into analysis prompts with no output size guidance; model loops and produces megabytes of text; missing max_tokens cap on the completion call.","solutions":["Cap output size via max_tokens on the completion request","Prompt the model to produce concise JSON and split very large inputs into smaller requests","Truncate/chunk the resume or job text passed to analyze/generate prompts","If justified, raise MAX_JSON_CONTENT_SIZE in llm.py"],"exampleFix":"// before\nconst response = await router.acompletion({ ...kwargs });\n// after\nconst response = await router.acompletion({ ...kwargs, max_tokens: 4096 });","handlingStrategy":"validation","validationCode":"MAX_JSON_CONTENT_SIZE = 1_000_000  # match llm.py\n\ndef content_size_ok(s: str) -> bool:\n    return len(s.encode('utf-8')) <= MAX_JSON_CONTENT_SIZE\n\n# check the raw response before parsing\nassert content_size_ok(response_text), \"LLM response exceeds JSON extraction size limit\"","typeGuard":null,"tryCatchPattern":"try:\n    data = await complete_json(prompt)\nexcept ValueError as e:\n    if \"Content too large\" in str(e):\n        data = await complete_json(prompt + \" Be concise; limit output to essential fields.\")\n    else:\n        raise","preventionTips":["Always set max_tokens on LLM calls to bound response size","Chunk large resumes/job descriptions into smaller analysis requests","Instruct models to emit concise JSON","Monitor response sizes in logs to tune MAX_JSON_CONTENT_SIZE"],"tags":["llm","json","size-limit","backend"],"backgroundTag":"payload-too-large","analyzedSha":"116f9cc3b00e1ac91734a6c2679bf41ea64a0edc","analyzedAt":"2026-08-28T22:51:40.999Z","schemaVersion":2},"datasetVersion":"2026-08-29T02:17:18.158Z"}