run-llama/llama_index · error · ImportError

OpenAIMultiModal is not installed. Please install it using `

Error message

OpenAIMultiModal is not installed. Please install it using `pip install llama-index-multi-modal-llms-openai`

What it means

Raised by RelevancyEvaluator.__init__ (multi-modal) when multi_modal_llm is not supplied and the optional integration package llama-index-llms-openai cannot be imported. The evaluator needs a default OpenAI LLM (OpenAIResponses, model gpt-4.1) to judge relevancy, so construction fails fast when the dependency is missing. It is an optional dependency of llama-index-core and is not installed by default.

Source

Thrown at llama-index-core/llama_index/core/evaluation/multi_modal/relevancy.py:73

            The template to use for refinement.

    """

    def __init__(
        self,
        multi_modal_llm: Optional[LLM] = None,
        raise_error: bool = False,
        eval_template: Union[str, BasePromptTemplate, None] = None,
        refine_template: Union[str, BasePromptTemplate, None] = None,
    ) -> None:
        """Init params."""
        if multi_modal_llm is None:
            try:
                from llama_index.llms.openai import (
                    OpenAIResponses,
                )  # pants: no-infer-dep
            except ImportError:
                raise ImportError(
                    "OpenAIMultiModal is not installed. "
                    "Please install it using `pip install llama-index-multi-modal-llms-openai`"
                )

            self._multi_modal_llm: LLM = OpenAIResponses(
                model="gpt-4.1", max_output_tokens=1000
            )
        else:
            self._multi_modal_llm = multi_modal_llm

        self._raise_error = raise_error

        self._eval_template: BasePromptTemplate
        if isinstance(eval_template, str):
            self._eval_template = PromptTemplate(eval_template)
        else:
            self._eval_template = eval_template or DEFAULT_EVAL_TEMPLATE

View on GitHub (pinned to afd0fef371)

Solutions

pip install llama-index-multi-modal-llms-openai (pulls llama-index-llms-openai)
Or pass your own LLM: MultiModalRelevancyEvaluator(multi_modal_llm=my_llm) to bypass the import entirely
If it must work offline, supply any LLM implementing llama_index.core.llms.LLM (e.g. a local Ollama or mock LLM)

Example fix

# before
evaluator = MultiModalRelevancyEvaluator()  # ImportError

# after
# pip install llama-index-multi-modal-llms-openai
from llama_index.llms.openai import OpenAIResponses
evaluator = MultiModalRelevancyEvaluator(
    multi_modal_llm=OpenAIResponses(model="gpt-4.1", max_output_tokens=1000)
)

Defensive patterns

Strategy: validation

Validate before calling

def has_openai_integration() -> bool:
    try:
        import llama_index.llms.openai  # noqa: F401
        return True
    except ImportError:
        return False

if not has_openai_integration():
    raise SystemExit("Install llama-index-multi-modal-llms-openai or pass multi_modal_llm")

from llama_index.core.evaluation.multi_modal.relevancy import MultiModalRelevancyEvaluator
evaluator = MultiModalRelevancyEvaluator()

Try / catch

try:
    evaluator = MultiModalRelevancyEvaluator()
except ImportError as e:
    if "llama-index-multi-modal-llms-openai" in str(e):
        logger.error("Missing optional integration; installing it or supply multi_modal_llm")
    raise

Prevention

Pin llama-index-multi-modal-llms-openai in requirements whenever you use multi-modal evaluators
Pass an explicit multi_modal_llm to make construction independent of optional packages
Smoke-test evaluator construction at app startup so dependency gaps surface before long eval runs

When it happens

Trigger: Constructing MultiModalRelevancyEvaluator (llama_index.core.evaluation.multi_modal.relevancy) without passing multi_modal_llm in an environment where `import llama_index.llms.openai` raises ImportError (package llama-index-multi-modal-llms-openai / llama-index-llms-openai not installed).

Common situations: Installing only llama-index-core or llama-index and assuming all evaluators work; fresh venv/CI image without the OpenAI extra; upgrading llama-index and hitting split-out integration packages (post-monorepo split each LLM lives in its own package).

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/20502c25758ac332. Report an issue: GitHub.