BerriAI/litellm · error · ReplicateError

Unable to parse response. Got={response_data["output"]}

Error message

Unable to parse response. Got={response_data["output"]}

What it means

Output-joining guard in the Replicate streaming poller: response_data['output'] could not be joined into a string (elements are not strings), so the streamed text cannot be reconstructed and a 422 is raised with the raw output payload.

Source

Thrown at litellm/llms/replicate/chat/handler.py:44

def handle_prediction_response_streaming(
    prediction_url, api_token, print_verbose, headers: dict, http_client: HTTPHandler
):
    previous_output = ""
    output_string = ""

    status = ""
    while True and (status not in ["succeeded", "failed", "canceled"]):
        time.sleep(REPLICATE_POLLING_DELAY_SECONDS)  # prevent being rate limited by replicate
        print_verbose(f"replicate: polling endpoint: {prediction_url}")
        response = http_client.get(prediction_url, headers=headers)
        if response.status_code == 200:
            response_data = response.json()
            status = response_data["status"]
            if "output" in response_data:
                try:
                    output_string = "".join(response_data["output"])
                except Exception:
                    raise ReplicateError(
                        status_code=422,
                        message="Unable to parse response. Got={}".format(response_data["output"]),
                        headers=response.headers,
                    )
                new_output = output_string[len(previous_output) :]
                print_verbose(f"New chunk: {new_output}")
                yield {"output": new_output, "status": status}
                previous_output = output_string
            status = response_data["status"]
            if status == "failed":
                replicate_error = response_data.get("error", "")
                raise ReplicateError(
                    status_code=400,
                    message=f"Error: {replicate_error}",
                    headers=response.headers,
                )
        else:
            # this can fail temporarily but it does not mean the replicate request failed, replicate request fails when status=="failed"

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. The Replicate response 'output' could not be parsed; log response_data["output"] to inspect its shape.
2. Check the Replicate model version's output schema.

Example fix

# verify the replicate deployment returns the expected output format.

When it happens

Trigger: Triggered when the Replicate chat response 'output' field cannot be parsed into the expected format.

Common situations: See trigger scenarios.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/898567e56a649bf9. Report an issue: GitHub.