deepset-ai/haystack · error

The Chat Generator returned no usable text to use as a conve

Error message

The Chat Generator returned no usable text to use as a conversation summary. Generator output: {result}.

What it means

_apply_summary raises RuntimeError when the summarization ChatGenerator's result contains no usable text (haystack/hooks/compaction/summarization.py:440). The hook inspects result['replies'] and needs a non-empty, non-whitespace reply to insert as the conversation summary; without it, compaction cannot proceed so it fails loudly rather than corrupting the conversation.

Source

Thrown at haystack/hooks/compaction/summarization.py:440

        before_tokens: int,
        token_counter: TokenCounter,
    ) -> tuple[list[ChatMessage], int]:
        """
        Swap the selected messages for the generated summary.

        :param messages: The conversation to compact, ordered oldest to newest.
        :param indices: The positions of the messages to replace with a summary.
        :param result: The Chat Generator's output, which should contain one usable summary.
        :param before_tokens: The already measured size of `messages`.
        :param token_counter: The counter used to verify that the summary actually shrinks the conversation.
        :returns: The conversation with the selected messages replaced by the summary, and its measured token count.
        :raises RuntimeError: If the generator returned no usable text, or if the swap did not make the conversation
            smaller, in which case keeping the raw messages is the better outcome.
        """
        replies = result.get("replies") or []
        text = replies[-1].text if replies else None
        if not text or not text.strip():
            raise RuntimeError(
                "The Chat Generator returned no usable text to use as a conversation summary. "
                f"Generator output: {result}."
            )

        summary = ChatMessage.from_user(
            text=f"<conversation_summary>\n{text.strip()}\n</conversation_summary>",
            meta={_COMPACTION_META_KEY: {"strategy": _STRATEGY, "summarized_messages": len(indices)}},
        )
        compacted = _replace_indices(messages=messages, indices=indices, summary=summary)
        after_tokens = token_counter.count(messages=compacted)
        if after_tokens >= before_tokens:
            raise RuntimeError(
                f"The generated summary did not reduce the conversation size ({before_tokens} tokens before and "
                f"{after_tokens} tokens after)."
            )
        return compacted, after_tokens

    def _report_failure(self, error: Exception) -> None:

View on GitHub (pinned to e318778c9b)

Solutions

  1. Inspect the logged result dict to see why replies were empty.
  2. Increase max_tokens / adjust the summary_instruction so the model reliably returns plain text.
  3. Check the generator's API/key configuration — auth or quota failures often surface as empty replies.
  4. Wrap compact() in error handling so a failed summary leaves the conversation intact and can be retried.

Example fix

// before
gen = OpenAIChatGenerator(model="gpt-4o-mini", generation_kwargs={"max_tokens": 1})
// after
gen = OpenAIChatGenerator(model="gpt-4o-mini", generation_kwargs={"max_tokens": 1024})
Defensive patterns

Strategy: try-catch

Validate before calling

def summary_will_be_usable(gen) -> bool:
    # smoke-test the generator returns text replies before wiring it into the compactor
    result = gen.run([ChatMessage.from_user("Reply with 'ok'.")])
    replies = result.get("replies") or []
    return bool(replies and replies[-1].text and replies[-1].text.strip())

Type guard

def has_usable_reply(result: dict) -> bool:
    replies = result.get("replies") or []
    return bool(replies) and bool((replies[-1].text or "").strip())

Try / catch

try:
    compacted = compactor.compact(messages, target_tokens, counter)
except RuntimeError as e:
    if "no usable text" in str(e):
        logger.warning("summarizer returned empty output; keeping raw messages")
        compacted = messages  # fallback
    else:
        raise

Prevention

When it happens

Trigger: The generator returns an empty replies list, replies with empty/whitespace-only text, or an unexpected result dict — e.g. the model refused, returned a tool call instead of text, or the generator errored into a non-text result.

Common situations: LLM refusal or safety stop producing empty output; misconfigured generator whose prompt yields a tool call; generator returning an error dict after an API failure; very small max_tokens truncating the reply to empty.

Related errors

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/9f0a264d9c241395. Report an issue: GitHub.