{"record":{"id":"0f4b54d9531bedbe","repo":"zylon-ai/private-gpt","slug":"model-server-is-not-available-or-request-failed","errorCode":null,"errorMessage":"Model server is not available or request failed.","messagePattern":"Model server is not available or request failed\\.","errorType":"exception","errorClass":"ModelNotAvailableError","httpStatus":null,"severity":"critical","filePath":"private_gpt/components/multimodality/audio_handler.py","lineNumber":499,"sourceCode":"\n                async def _call() -> Any:\n                    nonlocal count\n                    count += 1\n\n                    structured_chat = getattr(self._llm, \"astructured_chat\", None)\n                    if not callable(structured_chat):\n                        raise NotImplementedError(\n                            \"LLM does not support structured chat.\"\n                        )\n\n                    new_kwargs = kwargs.copy()\n                    new_kwargs[\"seed\"] = str(seed) + str(count)\n\n                    return await structured_chat(response_model, messages, **new_kwargs)\n\n                return await retry(_call)\n        except MODEL_NOT_AVAILABLE_EXCEPTION_TYPES as e:\n            raise ModelNotAvailableError(\n                \"Model server is not available or request failed.\"\n            ) from e\n        except Exception:\n            raise\n\n\nclass AudioProcessingWorkflow(Workflow):\n    def __init__(\n        self,\n        audio_multimodal_llm: LLM,\n        prompt_builder: PromptBuilderService | None = None,\n        callback_manager: CallbackManager | None = None,\n        timeout: float | None = 360000.0,\n        disable_validation: bool = False,\n        verbose: bool = False,\n        resource_manager: ResourceManager | None = None,\n        num_concurrent_runs: int | None = None,\n        max_workers: int = _DEFAULT_NUM_WORKERS,","sourceCodeStart":481,"sourceCodeEnd":517,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/multimodality/audio_handler.py#L481-L517","documentation":"Raised by the audio handler's outer except clause when the underlying LLM call fails with one of MODEL_NOT_AVAILABLE_EXCEPTION_TYPES (connection errors, timeouts, HTTP 404/503-style model-unavailable errors). It re-raises as the domain error ModelNotAvailableError with the original exception chained (`from e`), converting transport-level failures into a single predictable type after the retry loop has been exhausted. It means the model server backing the audio LLM could not be reached or refused the request.","triggerScenarios":"AudioProcessingWorkflow structured-chat call against a model server that is down, restarting, at a wrong URL/port, or returning not-found/unavailable for the model id; TLS/DNS failures; gateway returning 502/503; all retry attempts failing (the retry context with backoff and jitter wraps _call).","commonSituations":"Local inference server (vLLM/Ollama/llama.cpp) not started or still loading the model; wrong model id in settings; k8s pod restarts; firewall/service-mesh blocking the endpoint; auth rejected causing repeated failures.","solutions":["Verify the model server is up: curl its health/models endpoint from the app host.","Check the base_url, port, and model id in your LLM settings match the running server.","Inspect the chained cause (`ModelNotAvailableError.__cause__`) for the real transport error (timeout vs 404 vs auth).","If the server was starting, wait for model load and re-run; tune the retry/backoff settings if startup is slow."],"exampleFix":"# before: model url wrong\nsettings.llm_mode_openai.api_base = 'http://localhost:8000/v1'  # server on 8080\n\n# after\nsettings.llm_mode_openai.api_base = 'http://localhost:8080/v1'\n# curl http://localhost:8080/v1/models  # confirm model id before rerunning","handlingStrategy":"retry","validationCode":"import httpx\nresp = httpx.get(f'{base_url}/models', timeout=5)\nassert resp.status_code == 200, 'model server unreachable before running audio workflow'","typeGuard":null,"tryCatchPattern":"try:\n    result = await workflow.run(...)\nexcept ModelNotAvailableError as e:\n    cause = e.__cause__  # real transport error\n    if is_transient(cause):\n        await asyncio.sleep(backoff); result = await workflow.run(...)\n    else:\n        raise","preventionTips":["Add a readiness probe against the model server's /models endpoint before enabling audio flows.","Keep base_url and model id in env-validated config; inspect ModelNotAvailableError.__cause__ to distinguish DNS/timeout/404/auth failures."],"tags":["network","llm","audio","availability","retry"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}