{"record":{"id":"85981adcaaf4f791","repo":"abi/screenshot-to-code","slug":"inference-timed-out","errorCode":null,"errorMessage":"Inference timed out","messagePattern":"Inference timed out","errorType":"exception","errorClass":"TimeoutError","httpStatus":null,"severity":"error","filePath":"backend/image_generation/replicate.py","lineNumber":73,"sourceCode":"    for _ in range(MAX_POLLS):\n        await asyncio.sleep(POLL_INTERVAL_SECONDS)\n        status_response = await client.get(status_check_url, headers=headers)\n        status_response.raise_for_status()\n        status_response_raw: Any = status_response.json()\n        if not isinstance(status_response_raw, dict):\n            raise ValueError(\"Invalid prediction status response.\")\n        status_response_json = cast(dict[str, Any], status_response_raw)\n\n        status = status_response_json.get(\"status\")\n        if status == \"succeeded\":\n            return cast(dict[str, Any], status_response_json)\n        if status == \"error\":\n            error_message = str(status_response_json.get(\"error\", \"Unknown error\"))\n            raise ValueError(f\"Inference errored out: {error_message}\")\n        if status == \"failed\":\n            raise ValueError(\"Inference failed\")\n\n    raise TimeoutError(\"Inference timed out\")\n\n\nasync def _run_prediction(\n    endpoint_url: str, payload: dict[str, Any], api_token: str\n) -> Any:\n    headers = _build_headers(api_token)\n\n    async with httpx.AsyncClient() as client:\n        try:\n            response = await client.post(endpoint_url, headers=headers, json=payload)\n            response.raise_for_status()\n            response_json = response.json()\n            if not isinstance(response_json, dict):\n                raise ValueError(\"Invalid prediction creation response.\")\n\n            prediction_id = _extract_prediction_id(response_json)\n            final_response = await _poll_prediction(client, prediction_id, headers)\n            return final_response.get(\"output\")","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/abi/screenshot-to-code/blob/d026163f586dfa8c5c10d28c36edd59a9d3b0e88/backend/image_generation/replicate.py#L55-L91","documentation":"TimeoutError raised at the end of _poll_prediction: the loop exhausted MAX_POLLS iterations (each sleeping POLL_INTERVAL_SECONDS then GETting /predictions/{id}) without the status ever reaching succeeded/error/failed. The prediction is still \"starting\" or \"processing\" — Replicate just didn't finish within the client's polling budget.","triggerScenarios":"Long-running models (video generation, big batches) whose queue+inference time exceeds MAX_POLLS × POLL_INTERVAL_SECONDS; Replicate queue congestion during peak load; predictions stuck in \"starting\" due to cold boots.","commonSituations":"Calling slow models with the default poll budget; weekend/peak Replicate congestion; the creation response succeeded so the caller assumes inference itself will be fast.","solutions":["Raise MAX_POLLS or POLL_INTERVAL_SECONDS in the replicate module to cover the model's worst-case runtime.","Retry — the prediction may still succeed server-side; or query the prediction id directly to reuse it instead of re-paying.","Switch to a faster/smaller model version if latency budgets matter.","Check Replicate status for queue delays before assuming a code bug."],"exampleFix":"# before\nMAX_POLLS = 60\nPOLL_INTERVAL_SECONDS = 2.5\n\n# after — budget for slow models\nMAX_POLLS = 120\nPOLL_INTERVAL_SECONDS = 5.0","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    output = await call_replicate_model(model, input, token)\nexcept TimeoutError as e:\n    if str(e) == \"Inference timed out\":\n        # poll budget exhausted; prediction may still finish server-side\n        log.warning(\"Replicate poll budget exceeded for %s\", model)\n    raise","preventionTips":["Size MAX_POLLS x POLL_INTERVAL_SECONDS to the model's worst-case runtime (video models need minutes)","Distinguish this from \"Request timed out\" — that one is a single HTTP round-trip timeout","Prefer faster model versions when latency matters"],"tags":["replicate","timeout","image-generation","polling"],"backgroundTag":null,"analyzedSha":"d026163f586dfa8c5c10d28c36edd59a9d3b0e88","analyzedAt":"2026-08-14T22:02:06.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}