deepset-ai/haystack · error · RuntimeError

All {len(self.chat_generators)} chat generators failed. Last

Error message

All {len(self.chat_generators)} chat generators failed. Last error: {last_error}. Failed chat generators: [{failed_names}]

What it means

FallbackChatGenerator.run() tries each configured chat generator in order and collects failures. When every generator raised, it raises RuntimeError summarizing how many failed, the last underlying error, and the names of the failed generators. The root causes are the innermost errors of the wrapped generators.

Source

Thrown at haystack/components/generators/chat/fallback.py:206

                        "successful_chat_generator_class": gen_name,
                        "total_attempts": idx + 1,
                        "failed_chat_generators": failed,
                    }
                )
                return {"replies": replies, "meta": meta}
            except Exception as e:  # noqa: BLE001 - fallback logic should handle any exception
                logger.warning(
                    "ChatGenerator {chat_generator} failed with error: {error}", chat_generator=gen_name, error=e
                )
                failed.append(gen_name)
                last_error = e

        failed_names = ", ".join(failed)
        msg = (
            f"All {len(self.chat_generators)} chat generators failed. "
            f"Last error: {last_error}. Failed chat generators: [{failed_names}]"
        )
        raise RuntimeError(msg)

    @component.output_types(replies=list[ChatMessage], meta=dict[str, Any])
    async def run_async(
        self,
        messages: list[ChatMessage] | str,
        generation_kwargs: dict[str, Any] | None = None,
        tools: ToolsType | None = None,
        streaming_callback: StreamingCallbackT | None = None,
    ) -> dict[str, list[ChatMessage] | dict[str, Any]]:
        """
        Asynchronously execute chat generators sequentially until one succeeds.

        :param messages: The conversation history as a list of ChatMessage instances.
        :param generation_kwargs: Optional parameters for the chat generator (e.g., temperature, max_tokens).
        :param tools: A list of Tool and/or Toolset objects, or a single Toolset for function calling capabilities.
        :param streaming_callback: Optional callable for handling streaming responses.
        :returns: A dictionary with:
            - "replies": Generated ChatMessage instances from the first successful generator.

View on GitHub (pinned to e318778c9b)

Solutions

1. Read the 'Last error:' section of the message — it contains the underlying exception to fix first
2. Check API keys, quotas, and model/deployment names for every wrapped generator
3. Verify network/proxy access to each provider endpoint
4. Add more diverse fallback generators (different providers) so a single outage doesn't exhaust the list

Example fix

// before
try:
    res = fallback.run(messages=msgs)
except RuntimeError as e:
    print(e)  # inspect 'Last error' and failed generator names
// after
generator = FallbackChatGenerator([AzureChatGenerator(model="gpt-4o"), OpenAIChatGenerator(model="gpt-4o-mini")])
# ensure at least one generator has valid, independently billed credentials

# preflight: check each generator's client config before pipeline use
for g in fallback.chat_generators:
    d = g.to_dict()
    assert d.get("init_parameters", {}).get("api_base_url") or d.get("init_parameters", {}).get("azure_endpoint")

try:
    result = fallback.run(messages=msgs)
except RuntimeError as e:
    # message lists count, last error, and failed generator names
    logger.error("all fallback generators failed: %s", e)
    raise

When it happens

Trigger: Calling `run(messages=...)` when all wrapped generators raise (e.g. invalid API keys, exhausted quotas, network failures, misconfigured models) — including the case of a single generator that fails, since 'all' then means that one.

Common situations: Expired or wrong API keys for every provider in the list; rate limits hitting all generators simultaneously; all configured deployments deleted/renamed on the provider side; single-generator fallbacks used as a plain wrapper whose backend is down.

Related errors

• 'chat_generators' must be a non-empty list

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/061790c9fe181052. Report an issue: GitHub.