zylon-ai/private-gpt · critical · ModelNotAvailableError

Model server is not available or request failed.

Error message

Model server is not available or request failed.

What it means

Raised by the audio handler's outer except clause when the underlying LLM call fails with one of MODEL_NOT_AVAILABLE_EXCEPTION_TYPES (connection errors, timeouts, HTTP 404/503-style model-unavailable errors). It re-raises as the domain error ModelNotAvailableError with the original exception chained (`from e`), converting transport-level failures into a single predictable type after the retry loop has been exhausted. It means the model server backing the audio LLM could not be reached or refused the request.

Source

Thrown at private_gpt/components/multimodality/audio_handler.py:499

                async def _call() -> Any:
                    nonlocal count
                    count += 1

                    structured_chat = getattr(self._llm, "astructured_chat", None)
                    if not callable(structured_chat):
                        raise NotImplementedError(
                            "LLM does not support structured chat."
                        )

                    new_kwargs = kwargs.copy()
                    new_kwargs["seed"] = str(seed) + str(count)

                    return await structured_chat(response_model, messages, **new_kwargs)

                return await retry(_call)
        except MODEL_NOT_AVAILABLE_EXCEPTION_TYPES as e:
            raise ModelNotAvailableError(
                "Model server is not available or request failed."
            ) from e
        except Exception:
            raise


class AudioProcessingWorkflow(Workflow):
    def __init__(
        self,
        audio_multimodal_llm: LLM,
        prompt_builder: PromptBuilderService | None = None,
        callback_manager: CallbackManager | None = None,
        timeout: float | None = 360000.0,
        disable_validation: bool = False,
        verbose: bool = False,
        resource_manager: ResourceManager | None = None,
        num_concurrent_runs: int | None = None,
        max_workers: int = _DEFAULT_NUM_WORKERS,

View on GitHub (pinned to 4a030776a3)

Solutions

1. Verify the model server is up: curl its health/models endpoint from the app host.
2. Check the base_url, port, and model id in your LLM settings match the running server.
3. Inspect the chained cause (`ModelNotAvailableError.__cause__`) for the real transport error (timeout vs 404 vs auth).
4. If the server was starting, wait for model load and re-run; tune the retry/backoff settings if startup is slow.

Example fix

# before: model url wrong
settings.llm_mode_openai.api_base = 'http://localhost:8000/v1'  # server on 8080

# after
settings.llm_mode_openai.api_base = 'http://localhost:8080/v1'
# curl http://localhost:8080/v1/models  # confirm model id before rerunning

import httpx
resp = httpx.get(f'{base_url}/models', timeout=5)
assert resp.status_code == 200, 'model server unreachable before running audio workflow'

try:
    result = await workflow.run(...)
except ModelNotAvailableError as e:
    cause = e.__cause__  # real transport error
    if is_transient(cause):
        await asyncio.sleep(backoff); result = await workflow.run(...)
    else:
        raise

When it happens

Trigger: AudioProcessingWorkflow structured-chat call against a model server that is down, restarting, at a wrong URL/port, or returning not-found/unavailable for the model id; TLS/DNS failures; gateway returning 502/503; all retry attempts failing (the retry context with backoff and jitter wraps _call).

Common situations: Local inference server (vLLM/Ollama/llama.cpp) not started or still loading the model; wrong model id in settings; k8s pod restarts; firewall/service-mesh blocking the endpoint; auth rejected causing repeated failures.

Related errors

• Model server is not available or request failed.
• Audio blocks found but no audio-capable LLM provided.
• Failed to describe audio in the message.
• LLM does not support structured chat.
• Failed to generate SQL query

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/0f4b54d9531bedbe. Report an issue: GitHub.