abi/screenshot-to-code · error · TimeoutError

Inference timed out

Error message

Inference timed out

What it means

TimeoutError raised at the end of _poll_prediction: the loop exhausted MAX_POLLS iterations (each sleeping POLL_INTERVAL_SECONDS then GETting /predictions/{id}) without the status ever reaching succeeded/error/failed. The prediction is still "starting" or "processing" — Replicate just didn't finish within the client's polling budget.

Source

Thrown at backend/image_generation/replicate.py:73

    for _ in range(MAX_POLLS):
        await asyncio.sleep(POLL_INTERVAL_SECONDS)
        status_response = await client.get(status_check_url, headers=headers)
        status_response.raise_for_status()
        status_response_raw: Any = status_response.json()
        if not isinstance(status_response_raw, dict):
            raise ValueError("Invalid prediction status response.")
        status_response_json = cast(dict[str, Any], status_response_raw)

        status = status_response_json.get("status")
        if status == "succeeded":
            return cast(dict[str, Any], status_response_json)
        if status == "error":
            error_message = str(status_response_json.get("error", "Unknown error"))
            raise ValueError(f"Inference errored out: {error_message}")
        if status == "failed":
            raise ValueError("Inference failed")

    raise TimeoutError("Inference timed out")


async def _run_prediction(
    endpoint_url: str, payload: dict[str, Any], api_token: str
) -> Any:
    headers = _build_headers(api_token)

    async with httpx.AsyncClient() as client:
        try:
            response = await client.post(endpoint_url, headers=headers, json=payload)
            response.raise_for_status()
            response_json = response.json()
            if not isinstance(response_json, dict):
                raise ValueError("Invalid prediction creation response.")

            prediction_id = _extract_prediction_id(response_json)
            final_response = await _poll_prediction(client, prediction_id, headers)
            return final_response.get("output")

View on GitHub (pinned to d026163f58)

Solutions

1. Raise MAX_POLLS or POLL_INTERVAL_SECONDS in the replicate module to cover the model's worst-case runtime.
2. Retry — the prediction may still succeed server-side; or query the prediction id directly to reuse it instead of re-paying.
3. Switch to a faster/smaller model version if latency budgets matter.
4. Check Replicate status for queue delays before assuming a code bug.

Example fix

# before
MAX_POLLS = 60
POLL_INTERVAL_SECONDS = 2.5

# after — budget for slow models
MAX_POLLS = 120
POLL_INTERVAL_SECONDS = 5.0

try:
    output = await call_replicate_model(model, input, token)
except TimeoutError as e:
    if str(e) == "Inference timed out":
        # poll budget exhausted; prediction may still finish server-side
        log.warning("Replicate poll budget exceeded for %s", model)
    raise

When it happens

Trigger: Long-running models (video generation, big batches) whose queue+inference time exceeds MAX_POLLS × POLL_INTERVAL_SECONDS; Replicate queue congestion during peak load; predictions stuck in "starting" due to cold boots.

Common situations: Calling slow models with the default poll budget; weekend/peak Replicate congestion; the creation response succeeded so the caller assumes inference itself will be fast.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Invalid prediction status response.
• Prediction ID not found in initial response.
• Inference errored out: {error_message}
• Inference failed
• Invalid prediction creation response.

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/85981adcaaf4f791. Report an issue: GitHub.