{"record":{"id":"c7209e50b5d95b8d","repo":"666ghj/MiroFish","slug":"llm-json-output-was-truncated-at-the-token-limit","errorCode":null,"errorMessage":"LLM JSON output was truncated at the token limit","messagePattern":"LLM JSON output was truncated at the token limit","errorType":"exception","errorClass":"LLMResponseError","httpStatus":null,"severity":"error","filePath":"backend/app/utils/llm_client.py","lineNumber":243,"sourceCode":"                    \"retrying content generation%s\",\n                    error.finish_reason or \"unknown\",\n                    \" without an output token cap\" if had_token_cap else \"\",\n                )\n\n        if last_error is not None:  # pragma: no cover - defensive loop guard\n            raise last_error\n        raise LLMResponseError(\"LLM did not produce a JSON response\")\n\n    @staticmethod\n    def _parse_json_response(response: Any) -> Dict[str, Any]:\n        choices = getattr(response, \"choices\", None) or []\n        if not choices:\n            raise LLMResponseError(\"LLM returned no choices\")\n\n        choice = choices[0]\n        finish_reason = getattr(choice, \"finish_reason\", None)\n        if finish_reason == \"length\":\n            raise LLMResponseError(\n                \"LLM JSON output was truncated at the token limit\",\n                finish_reason=finish_reason,\n            )\n        if finish_reason not in {None, \"stop\"}:\n            raise LLMResponseError(\n                f\"LLM JSON generation stopped unexpectedly ({finish_reason})\",\n                finish_reason=finish_reason,\n            )\n\n        content = _clean_chat_text(extract_chat_completion_text(response))\n        if not content:\n            raise LLMResponseError(\n                \"LLM returned empty JSON content\",\n                finish_reason=finish_reason,\n            )\n\n        try:\n            value = json.loads(content)","sourceCodeStart":225,"sourceCodeEnd":261,"githubUrl":"https://github.com/666ghj/MiroFish/blob/b5b53acc57189a4a42e44a23e149dc655c98fe82/backend/app/utils/llm_client.py#L225-L261","documentation":"LLMResponseError raised when choices[0].finish_reason == 'length': the JSON generation hit the max output token cap and was cut off mid-JSON, so the content cannot be trusted or parsed. The client deliberately does not repair truncated JSON (it never invents data), so it fails with the finish_reason attached.","triggerScenarios":"Requesting structured JSON output (e.g. via _create_completion with max_tokens) where the model's JSON answer needs more tokens than the cap allows — large schemas, long arrays, or a max_tokens set too small for the requested structure.","commonSituations":"Tight max_tokens defaults to control cost, models that produce verbose JSON (long sub-question lists, big entity summaries), or prompts asking for many items in one response.","solutions":["Increase max_tokens (or remove the output cap) for the affected call","Shrink the requested output: fewer items, shorter fields, or split the request into multiple smaller JSON calls","Switch to a model with a larger output window if the schema genuinely needs it","Do not attempt to 'fix up' the truncated JSON string — the code intentionally rejects repaired JSON"],"exampleFix":"# before\nresult = client.generate_json(messages, max_tokens=512)  # truncated -> LLMResponseError\n\n# after\nresult = client.generate_json(messages, max_tokens=4096)","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    result = client.generate_json(messages, max_tokens=512)\nexcept LLMResponseError as e:\n    if e.finish_reason == \"length\":\n        result = client.generate_json(messages, max_tokens=4096)  # retry with larger cap\n    else:\n        raise","preventionTips":["Budget max_tokens to the expected JSON size times a safety factor","Split large structured outputs across multiple smaller calls","Never repair truncated JSON — retry with a bigger cap instead"],"tags":["llm","json","token-limit","truncation"],"backgroundTag":null,"analyzedSha":"b5b53acc57189a4a42e44a23e149dc655c98fe82","analyzedAt":"2026-08-14T22:29:33.146Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}