{"record":{"id":"9f0a264d9c241395","repo":"deepset-ai/haystack","slug":"the-chat-generator-returned-no-usable-text-to-use","errorCode":null,"errorMessage":"The Chat Generator returned no usable text to use as a conversation summary. Generator output: {result}.","messagePattern":"The Chat Generator returned no usable text to use as a conversation summary\\. Generator output: (.+?)\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/hooks/compaction/summarization.py","lineNumber":440,"sourceCode":"        before_tokens: int,\n        token_counter: TokenCounter,\n    ) -> tuple[list[ChatMessage], int]:\n        \"\"\"\n        Swap the selected messages for the generated summary.\n\n        :param messages: The conversation to compact, ordered oldest to newest.\n        :param indices: The positions of the messages to replace with a summary.\n        :param result: The Chat Generator's output, which should contain one usable summary.\n        :param before_tokens: The already measured size of `messages`.\n        :param token_counter: The counter used to verify that the summary actually shrinks the conversation.\n        :returns: The conversation with the selected messages replaced by the summary, and its measured token count.\n        :raises RuntimeError: If the generator returned no usable text, or if the swap did not make the conversation\n            smaller, in which case keeping the raw messages is the better outcome.\n        \"\"\"\n        replies = result.get(\"replies\") or []\n        text = replies[-1].text if replies else None\n        if not text or not text.strip():\n            raise RuntimeError(\n                \"The Chat Generator returned no usable text to use as a conversation summary. \"\n                f\"Generator output: {result}.\"\n            )\n\n        summary = ChatMessage.from_user(\n            text=f\"<conversation_summary>\\n{text.strip()}\\n</conversation_summary>\",\n            meta={_COMPACTION_META_KEY: {\"strategy\": _STRATEGY, \"summarized_messages\": len(indices)}},\n        )\n        compacted = _replace_indices(messages=messages, indices=indices, summary=summary)\n        after_tokens = token_counter.count(messages=compacted)\n        if after_tokens >= before_tokens:\n            raise RuntimeError(\n                f\"The generated summary did not reduce the conversation size ({before_tokens} tokens before and \"\n                f\"{after_tokens} tokens after).\"\n            )\n        return compacted, after_tokens\n\n    def _report_failure(self, error: Exception) -> None:","sourceCodeStart":422,"sourceCodeEnd":458,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/hooks/compaction/summarization.py#L422-L458","documentation":"_apply_summary raises RuntimeError when the summarization ChatGenerator's result contains no usable text (haystack/hooks/compaction/summarization.py:440). The hook inspects result['replies'] and needs a non-empty, non-whitespace reply to insert as the conversation summary; without it, compaction cannot proceed so it fails loudly rather than corrupting the conversation.","triggerScenarios":"The generator returns an empty replies list, replies with empty/whitespace-only text, or an unexpected result dict — e.g. the model refused, returned a tool call instead of text, or the generator errored into a non-text result.","commonSituations":"LLM refusal or safety stop producing empty output; misconfigured generator whose prompt yields a tool call; generator returning an error dict after an API failure; very small max_tokens truncating the reply to empty.","solutions":["Inspect the logged result dict to see why replies were empty.","Increase max_tokens / adjust the summary_instruction so the model reliably returns plain text.","Check the generator's API/key configuration — auth or quota failures often surface as empty replies.","Wrap compact() in error handling so a failed summary leaves the conversation intact and can be retried."],"exampleFix":"// before\ngen = OpenAIChatGenerator(model=\"gpt-4o-mini\", generation_kwargs={\"max_tokens\": 1})\n// after\ngen = OpenAIChatGenerator(model=\"gpt-4o-mini\", generation_kwargs={\"max_tokens\": 1024})","handlingStrategy":"try-catch","validationCode":"def summary_will_be_usable(gen) -> bool:\n    # smoke-test the generator returns text replies before wiring it into the compactor\n    result = gen.run([ChatMessage.from_user(\"Reply with 'ok'.\")])\n    replies = result.get(\"replies\") or []\n    return bool(replies and replies[-1].text and replies[-1].text.strip())","typeGuard":"def has_usable_reply(result: dict) -> bool:\n    replies = result.get(\"replies\") or []\n    return bool(replies) and bool((replies[-1].text or \"\").strip())","tryCatchPattern":"try:\n    compacted = compactor.compact(messages, target_tokens, counter)\nexcept RuntimeError as e:\n    if \"no usable text\" in str(e):\n        logger.warning(\"summarizer returned empty output; keeping raw messages\")\n        compacted = messages  # fallback\n    else:\n        raise","preventionTips":["Set adequate max_tokens on the summarization generator","Test the generator's replies format once at startup","Handle model refusals by rephrasing summary_instruction","Check API key/quota health — failures often surface as empty replies"],"tags":["llm","empty-output","summary","runtime","compaction"],"backgroundTag":"llm-empty-response","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}