{"record":{"id":"08f08ed78b0b0c34","repo":"docling-project/docling","slug":"could-not-find-assistant-response-in-decoded-text","errorCode":null,"errorMessage":"Could not find assistant response in decoded text","messagePattern":"Could not find assistant response in decoded text","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"docling/models/stages/chart_extraction/granite_vision.py","lineNumber":319,"sourceCode":"            except Exception as e:\n                _log.error(f\"Failed to extract DataFrame for image {i}: {e}\")\n                chart_data.append(None)\n\n        return chart_data\n\n    def _extract_csv_to_dataframe(self, decoded_text: str) -> pd.DataFrame:\n        \"\"\"\n        Extract CSV content from decoded text and convert to DataFrame.\n\n        Handles:\n        - Chat format with <|assistant|> tags\n        - Nested code blocks (```csv ``` inside ```)\n        - Various CSV formatting issues\n        \"\"\"\n        # Extract the assistant's response\n        assistant_match = re.search(r\"<\\|assistant\\|>\\s*(.*)\", decoded_text, re.DOTALL)\n        if not assistant_match:\n            raise ValueError(\"Could not find assistant response in decoded text\")\n\n        assistant_response = assistant_match.group(1).strip()\n\n        # Extract the first CSV code block (```csv ... ```)\n        csv_match = re.search(r\"```csv\\s*\\n(.*?)\\n```\", assistant_response, re.DOTALL)\n        if csv_match:\n            csv_content = csv_match.group(1).strip()\n        else:\n            # Fallback: take content up to first <|end_of_text|> and strip code block markers\n            csv_content = assistant_response.split(\"<|end_of_text|>\")[0].strip()\n            csv_content = re.sub(r\"^```+(?:csv)?\\s*\", \"\", csv_content)\n            csv_content = re.sub(r\"```+\\s*$\", \"\", csv_content)\n            csv_content = csv_content.strip()\n\n        try:\n            dataframe = pd.read_csv(StringIO(csv_content), header=None)\n            return dataframe\n        except Exception as e:","sourceCodeStart":301,"sourceCodeEnd":337,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/stages/chart_extraction/granite_vision.py#L301-L337","documentation":"The Granite chart-extraction model decodes the LLM output and expects a chat-format response containing an `<|assistant|>` tag. This ValueError is raised when the regex `<\\|assistant\\|>\\s*(.*)` finds no match in the decoded text, meaning the model's raw output does not follow the expected chat template (e.g. it is empty, truncated, or a plain completion without chat markers). It almost always indicates a generation/pipeline issue rather than a document problem.","triggerScenarios":"Calling the granite-vision chart-to-CSV stage when the underlying Granite VLM returns output without any `<|assistant|>` marker: truncated generation (max_new_tokens hit early), a model revision that changed the chat template, wrong processor/chat-template application, or empty decoder output.","commonSituations":"Using the chart extraction stage with a newer/older granite-vision checkpoint whose prompt format differs; low max token limits producing cut-off output; passing a non-chat-formatted prompt; decoding a batch where the assistant turn was dropped.","solutions":["Inspect the decoded text (log it) to see what the model actually returned — empty, truncated, or differently formatted.","Verify the prompt was built with the model's chat template (apply_chat_template with add_generation_prompt=True) so the model emits `<|assistant|>`.","Increase generation limits (max_new_tokens) so the assistant response is not truncated before it begins.","Pin the granite-vision model revision known to work with this stage; if the checkpoint changed its template, update the regex/extraction to match."],"exampleFix":"# before\nassistant_match = re.search(r\"<\\|assistant\\|>\\s*(.*)\", decoded_text, re.DOTALL)\nif not assistant_match:\n    raise ValueError(\"Could not find assistant response in decoded text\")\n\n# after — fall back to the whole decoded text when no chat marker is present\nassistant_match = re.search(r\"<\\|assistant\\|>\\s*(.*)\", decoded_text, re.DOTALL)\nif assistant_match:\n    assistant_response = assistant_match.group(1).strip()\nelse:\n    _log.warning(\"No <|assistant|> tag in Granite output; using raw decoded text\")\n    assistant_response = decoded_text.strip()","handlingStrategy":"try-catch","validationCode":"import re\n\ndef has_assistant_response(decoded_text: str) -> bool:\n    return re.search(r\"<\\|assistant\\|>\\s*(.)\", decoded_text, re.DOTALL) is not None","typeGuard":null,"tryCatchPattern":"try:\n    df = stage._extract_csv_to_dataframe(decoded)\nexcept ValueError as err:\n    if \"assistant response\" in str(err):\n        logger.warning(\"Granite output lacked assistant tag; skipping chart: %s\", decoded[:200])\n        continue  # skip this chart rather than fail the document\n    raise","preventionTips":["Log raw model outputs at DEBUG during development so template mismatches are visible.","Pin the granite-vision model revision used in production.","Set generation limits high enough that the assistant turn is always produced."],"tags":["chart-extraction","granite-vision","llm-output","parsing"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}