BerriAI/litellm · error · ValueError

Unable to generate response

Error message

Unable to generate response

What it means

Catch-all ValueError in the async non-Gemini completion flow: the awaited prediction did not yield a usable completion response. It wraps failures from predict_async / response parsing when no more specific error applies.

Source

Thrown at litellm/llms/vertex_ai/vertex_ai_non_gemini.py:671

    elif mode == "private":
        if instances is None:
            raise ValueError("Instances are required for private endpoint")
        stream = optional_params.pop("stream", None)
        _ = instances[0].pop("stream", None)
        request_str += f"llm_model.predict_async(instances={instances})\n"
        response_obj = await llm_model.predict_async(
            instances=instances,
        )
        response = response_obj.predictions
        completion_response = response[0]
        if isinstance(completion_response, str) and "\nOutput:\n" in completion_response:
            completion_response = completion_response.split("\nOutput:\n", 1)[1]
        if stream:
            response = TextStreamer(completion_response)

    if response is None:
        raise ValueError("Unable to generate response")

    logging_obj.post_call(input=prompt, api_key=None, original_response=response)

    streamwrapper: Final = CustomStreamWrapper(
        completion_stream=response,
        model=model,
        custom_llm_provider="vertex_ai",
        logging_obj=logging_obj,
    )

    return streamwrapper

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Check upstream logs / verbose output (litellm.set_verbose=True) for the underlying cause of the failed generation.
2. Verify the endpoint, credentials, and request payload; retry the request.

When it happens

Trigger: Thrown at litellm/llms/vertex_ai/vertex_ai_non_gemini.py:671 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/0771395f3f86b3ad. Report an issue: GitHub.