zylon-ai/private-gpt · error · ImportError

OpenAI-like LLM dependencies are not installed. Install with

Error message

OpenAI-like LLM dependencies are not installed. Install with `uv sync --inexact --extra llm-openai-compatible`.

What it means

ImportError raised by the lazy loader in the OpenAI-like completions factory: importing PatchedOpenAILikeLLM failed because the optional 'llm-openai-compatible' extra is not installed. This factory serves OpenAI-compatible endpoints such as vLLM and Ollama, which need the llama-index openai-like integration packages.

Source

Thrown at private_gpt/components/llm/factories/completions/openailike.py:18

from typing import Any

from llama_index.core.llms import LLM

from private_gpt.components.llm.factories.base import LLMFactory
from private_gpt.components.llm.prompt_helper import get_prompt_style
from private_gpt.components.llm.tokenizers.tokenizer_base import TokenizerBase
from private_gpt.settings.settings import LLMModelConfig, Settings
from private_gpt.utils.dependencies import format_missing_dependency_message


def _load_patched_openai_like() -> type[Any]:
    try:
        from private_gpt.components.llm.custom.openailike import PatchedOpenAILikeLLM

        return PatchedOpenAILikeLLM
    except ImportError as e:
        raise ImportError(
            format_missing_dependency_message(
                "OpenAI-like LLM",
                extras="llm-openai-compatible",
            )
        ) from e


class OpenAILikeCompletionsFactory(LLMFactory):
    """Chat Completions factory for OpenAI-compatible endpoints (vLLM, Ollama, etc.)."""

    def __init__(self, settings: Settings):
        super().__init__(settings)
        self.openai_settings = settings.openai

    def _create_llm(
        self, model_config: LLMModelConfig, tokenizer: TokenizerBase | None = None
    ) -> tuple[LLM, str | None]:
        PatchedOpenAILikeLLM = _load_patched_openai_like()

View on GitHub (pinned to 4a030776a3)

Solutions

Run `uv sync --inexact --extra llm-openai-compatible`.
Otherwise install the packages listed under the llm-openai-compatible extra in pyproject.toml.
Rebuild deployment images with the extra included.

Example fix

# before
uv sync

# after
uv sync --inexact --extra llm-openai-compatible

Defensive patterns

Strategy: validation

Validate before calling

def openai_compatible_extra_available() -> bool:
    try:
        from llama_index.llms.openai_like import OpenAILike  # noqa: F401
        return True
    except ImportError:
        return False

Try / catch

try:
    factory = registry.get_factory("openai-like")
except ImportError as e:
    if 'llm-openai-compatible' in str(e):
        raise SystemExit('Install extras: uv sync --inexact --extra llm-openai-compatible') from e
    raise

Prevention

Map every LLM mode you configure to its required extra in a setup checklist.
Use a lockfile-based sync with all needed extras in Docker builds.
Smoke-test the configured mode in a healthcheck endpoint after deploy.

When it happens

Trigger: Setting the LLM mode to an OpenAI-compatible backend (vLLM, Ollama, LM Studio, etc.) without the llm-openai-compatible extra installed.

Common situations: Self-hosted deployments pointing at a local vLLM/Ollama server on a machine where only core dependencies were synced; slim Docker images; switching from hosted OpenAI to local inference without adding the extra.

Related errors

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/cd6526307c6cfc99. Report an issue: GitHub.