{"record":{"id":"173f493039477a1f","repo":"deepset-ai/haystack","slug":"the-generated-summary-did-not-reduce-the-conversat","errorCode":null,"errorMessage":"The generated summary did not reduce the conversation size ({before_tokens} tokens before and {after_tokens} tokens after).","messagePattern":"The generated summary did not reduce the conversation size \\((.+?) tokens before and (.+?) tokens after\\)\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"haystack/hooks/compaction/summarization.py","lineNumber":452,"sourceCode":"        :raises RuntimeError: If the generator returned no usable text, or if the swap did not make the conversation\n            smaller, in which case keeping the raw messages is the better outcome.\n        \"\"\"\n        replies = result.get(\"replies\") or []\n        text = replies[-1].text if replies else None\n        if not text or not text.strip():\n            raise RuntimeError(\n                \"The Chat Generator returned no usable text to use as a conversation summary. \"\n                f\"Generator output: {result}.\"\n            )\n\n        summary = ChatMessage.from_user(\n            text=f\"<conversation_summary>\\n{text.strip()}\\n</conversation_summary>\",\n            meta={_COMPACTION_META_KEY: {\"strategy\": _STRATEGY, \"summarized_messages\": len(indices)}},\n        )\n        compacted = _replace_indices(messages=messages, indices=indices, summary=summary)\n        after_tokens = token_counter.count(messages=compacted)\n        if after_tokens >= before_tokens:\n            raise RuntimeError(\n                f\"The generated summary did not reduce the conversation size ({before_tokens} tokens before and \"\n                f\"{after_tokens} tokens after).\"\n            )\n        return compacted, after_tokens\n\n    def _report_failure(self, error: Exception) -> None:\n        \"\"\"Re-raise a failed summarization or log it, so whatever compacted successfully so far is still returned.\"\"\"\n        if self.raise_on_failure:\n            raise error\n        logger.warning(\n            \"Summarizing the conversation for context compaction failed; keeping the last successful result. \"\n            \"Error: {error}\",\n            error=error,\n        )\n\n    def warm_up(self) -> None:\n        \"\"\"Warm up the Chat Generator that writes summaries.\"\"\"\n        if hasattr(self.chat_generator, \"warm_up\"):","sourceCodeStart":434,"sourceCodeEnd":470,"githubUrl":"https://github.com/deepset-ai/haystack/blob/e318778c9bf60a1963e3b5f451359655dd696c30/haystack/hooks/compaction/summarization.py#L434-L470","documentation":"_apply_summary raises RuntimeError when the summary swap fails to shrink the conversation (haystack/hooks/compaction/summarization.py:452). It re-counts tokens after replacing the summarized messages with the summary and requires after_tokens < before_tokens; otherwise keeping the raw messages is strictly better, so the hook refuses the result.","triggerScenarios":"The generated summary is as long or longer than the messages it replaces — e.g. a verbose model, an instruction that doesn't ask for brevity, or a tiny number of summarized messages dominated by the <conversation_summary> wrapper overhead.","commonSituations":"Model ignoring length instructions; summarizing only a few short messages so overhead exceeds savings; summary_instruction prompting for detailed recaps; switching to a model that writes long outputs.","solutions":["Tighten summary_instruction to demand a terse summary (e.g. 'in at most 200 words, bullet points').","Reduce approximate_summary_tokens / generation max_tokens to cap summary length.","Raise compact_at so more messages get summarized at once, making the swap clearly smaller.","Catch the RuntimeError and fall back to the raw (uncompacted) conversation, as the hook intends."],"exampleFix":"// before\ncompactor = SummarizationCompactor(gen, summary_instruction=\"Summarize the conversation.\")\n// after\ncompactor = SummarizationCompactor(gen, summary_instruction=\"Summarize the conversation in at most 150 words of terse bullets.\")","handlingStrategy":"try-catch","validationCode":"def summary_instruction_is_terse() -> str:\n    return \"Summarize the conversation in at most 150 words using terse bullet points.\"\n# ensure generation_kwargs cap output too:\ngeneration_kwargs = {\"max_tokens\": 400}","typeGuard":null,"tryCatchPattern":"try:\n    compacted = compactor.compact(messages, target_tokens, counter)\nexcept RuntimeError as e:\n    if \"did not reduce\" in str(e):\n        logger.warning(\"summary too large (%s); keeping raw messages\", e)\n        compacted = messages  # hook semantics: raw messages are the better outcome\n    else:\n        raise","preventionTips":["Make summary_instruction explicitly request brevity","Cap summary length via max_tokens","Avoid summarizing very few/short messages where overhead dominates","Raise compact_at so more content is summarized per pass"],"tags":["llm","summary","token-budget","runtime","compaction"],"backgroundTag":"llm-output-does-not-meet-constraint","analyzedSha":"e318778c9bf60a1963e3b5f451359655dd696c30","analyzedAt":"2026-08-30T11:45:20.711Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}