abi/screenshot-to-code · error · ValueError

Inference failed

Error message

Inference failed

What it means

Raised in _poll_prediction when the prediction's status becomes "failed": Replicate terminated the prediction without the detailed "error" payload. Unlike the "error" status (which carries an error message), "failed" gives no diagnostics here, so this ValueError is the generic terminal-failure signal after MAX_POLLS rounds of polling.

Source

Thrown at backend/image_generation/replicate.py:71

    status_check_url = f"{REPLICATE_API_BASE_URL}/predictions/{prediction_id}"

    for _ in range(MAX_POLLS):
        await asyncio.sleep(POLL_INTERVAL_SECONDS)
        status_response = await client.get(status_check_url, headers=headers)
        status_response.raise_for_status()
        status_response_raw: Any = status_response.json()
        if not isinstance(status_response_raw, dict):
            raise ValueError("Invalid prediction status response.")
        status_response_json = cast(dict[str, Any], status_response_raw)

        status = status_response_json.get("status")
        if status == "succeeded":
            return cast(dict[str, Any], status_response_json)
        if status == "error":
            error_message = str(status_response_json.get("error", "Unknown error"))
            raise ValueError(f"Inference errored out: {error_message}")
        if status == "failed":
            raise ValueError("Inference failed")

    raise TimeoutError("Inference timed out")


async def _run_prediction(
    endpoint_url: str, payload: dict[str, Any], api_token: str
) -> Any:
    headers = _build_headers(api_token)

    async with httpx.AsyncClient() as client:
        try:
            response = await client.post(endpoint_url, headers=headers, json=payload)
            response.raise_for_status()
            response_json = response.json()
            if not isinstance(response_json, dict):
                raise ValueError("Invalid prediction creation response.")

            prediction_id = _extract_prediction_id(response_json)

View on GitHub (pinned to d026163f58)

Solutions

1. Retry the call once — "failed" is frequently transient infrastructure, unlike parameter errors.
2. Check the Replicate status page and the prediction in the Replicate dashboard (its logs often explain the failure).
3. Pin a stable model version if the failure repeats with the latest tag.
4. If persistent, test the same input via curl against the Replicate API to isolate client vs model.

for attempt in range(2):
    try:
        output = await call_replicate_model(model, input, token)
        break
    except ValueError as e:
        if str(e) == "Inference failed" and attempt == 0:
            continue  # status "failed" is often transient infra
        raise

When it happens

Trigger: A prediction that transitions to status "failed" on Replicate's side — commonly GPU/infra failures, canceled predictions, cold-start crashes, or model runtime exceptions that don't populate the error field.

Common situations: Transient Replicate infra hiccups; model version removed or deprecated mid-flight; overloaded models during peak times.

Related errors

• Inference errored out: {error_message}
• Prediction ID not found in initial response.
• Invalid prediction status response.
• Inference timed out
• Invalid prediction creation response.

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/4708a6a90274c196. Report an issue: GitHub.