mem0ai/mem0 · error · RuntimeError

Failed to generate response: {e}

Error message

Failed to generate response: {e}

What it means

RuntimeError raised by AWSBedrockLLM.generate_response as the outer wrapper around ANY exception during response generation — tool-enabled paths (_generate_with_tools) and standard paths (_generate_standard) both funnel here. The original exception is logged (logger.error) and appended to the message; the real cause is almost always the inner AWS error (throttle, model access, malformed messages, context length).

Source

Thrown at mem0/llms/aws_bedrock.py:468

            tools: List of tools for function calling
            tool_choice: Tool choice method
            stream: Whether to stream the response
            **kwargs: Additional parameters

        Returns:
            Generated response
        """
        try:
            if tools and self.supports_tools:
                # Use converse method for tool-enabled models
                return self._generate_with_tools(messages, tools, stream)
            else:
                # Use standard invoke_model method
                return self._generate_standard(messages, stream)

        except Exception as e:
            logger.error(f"Failed to generate response: {e}")
            raise RuntimeError(f"Failed to generate response: {e}")

    @staticmethod
    def _convert_tools_to_converse_format(tools: List[Dict]) -> List[Dict]:
        """Convert OpenAI-style tools to Converse API format."""
        if not tools:
            return []

        converse_tools = []
        for tool in tools:
            if tool.get("type") == "function" and "function" in tool:
                func = tool["function"]
                converse_tool = {
                    "toolSpec": {
                        "name": func["name"],
                        "description": func.get("description", ""),
                        "inputSchema": {
                            "json": func.get("parameters", {})
                        }

View on GitHub (pinned to 001c235229)

Solutions

1. Read the '{e}' portion of the message and the mem0 error log to identify the underlying AWS exception, then fix that specifically
2. For throttling: add exponential-backoff retry around Memory.add/search calls or reduce concurrency
3. For model access: enable the model in the Bedrock console for the region
4. For context-length errors: reduce prompt/history size or switch to a model with a larger context window
5. Update mem0ai — Bedrock integration fixes (tool conversion, streaming) land regularly

Example fix

// before
result = memory.add("user likes tea", user_id="a")  # RuntimeError: Failed to generate response: ThrottlingException

# after
import time
for attempt in range(5):
    try:
        result = memory.add("user likes tea", user_id="a")
        break
    except RuntimeError as e:
        if "Throttling" not in str(e) or attempt == 4:
            raise
        time.sleep(2 ** attempt)

# preflight before heavy use: verify model invocation works
resp = boto3.client("bedrock-runtime", region_name=region).invoke_model(
    modelId=model_id, body=b'{"prompt":"hi","max_tokens":1}')
assert resp["body"].read(), "model invocation failed"

import time

def generate_with_backoff(fn, *args, retries=5, **kw):
    for i in range(retries):
        try:
            return fn(*args, **kw)
        except RuntimeError as e:
            msg = str(e)
            transient = any(t in msg for t in ("Throttling", "ServiceUnavailable", "timeout", "Timeout"))
            if not transient or i == retries - 1:
                raise
            time.sleep(2 ** i)

When it happens

Trigger: Calling Memory.add/search which invokes the LLM; Bedrock ThrottlingException under load; ModelStreamErrorException or AccessDeniedException for a model the account lacks access to; context-length exceeded for the converse API; unsupported tool schema in _convert_tools_to_converse_format raising inside the tool path

Common situations: Production bursts hitting Bedrock TPS limits; account without model access granted for the specific model ID; malformed chat history (invalid role alternation) rejected by converse API; partial JSON responses from the model breaking downstream parsing in the tool loop.

Related errors

• The 'boto3' library is required. Please install it using 'pi
• Unknown provider_override '{explicit_provider}'. Valid provi
• Unknown provider in model: {model}
• AWS credentials not found. Please set AWS_ACCESS_KEY_ID, AWS
• Unauthorized access to Bedrock. Please ensure your AWS crede

AI-assisted analysis of mem0ai/mem0@001c235229 (2026-08-15). Data as JSON: /api/errors/1fbbc75b662385dc. Report an issue: GitHub.