run-llama/llama_index · error · ImportError

llama-index-llms-openai is not installed. Please install it

Error message

llama-index-llms-openai is not installed. Please install it using `pip install llama-index-multi-modal-llms-openai`

What it means

Raised by MultiModalFaithfulnessEvaluator.__init__ when no multi_modal_llm is supplied and the fallback import of llama_index.llms.openai (package llama-index-llms-openai) fails. Note the message's pip hint names llama-index-multi-modal-llms-openai, but the code actually imports from llama-index-llms-openai — install that package (or pass your own multi-modal LLM).

Source

Thrown at llama-index-core/llama_index/core/evaluation/multi_modal/faithfulness.py:94

            The template to use for refining the evaluation.

    """

    def __init__(
        self,
        multi_modal_llm: Optional[LLM] = None,
        raise_error: bool = False,
        eval_template: Union[str, BasePromptTemplate, None] = None,
        refine_template: Union[str, BasePromptTemplate, None] = None,
    ) -> None:
        """Init params."""
        if multi_modal_llm is None:
            try:
                from llama_index.llms.openai import (
                    OpenAIResponses,
                )  # pants: no-infer-dep
            except ImportError:
                raise ImportError(
                    "llama-index-llms-openai is not installed. "
                    "Please install it using `pip install llama-index-multi-modal-llms-openai`"
                )

            self._multi_modal_llm: LLM = OpenAIResponses(
                model="gpt-4.1", max_output_tokens=1000
            )
        else:
            self._multi_modal_llm = multi_modal_llm

        self._raise_error = raise_error

        self._eval_template: BasePromptTemplate
        if isinstance(eval_template, str):
            self._eval_template = PromptTemplate(eval_template)
        else:
            self._eval_template = eval_template or DEFAULT_EVAL_TEMPLATE

View on GitHub (pinned to afd0fef371)

Solutions

pip install llama-index-llms-openai (despite the message mentioning llama-index-multi-modal-llms-openai, the import resolves to the former; the extra llama-index-llms-openai[multi-modal] variant covers vision deps).
Or pass your own LLM: MultiModalFaithfulnessEvaluator(multi_modal_llm=your_mm_llm) to avoid the OpenAI fallback entirely.
Verify with python -c "from llama_index.llms.openai import OpenAIResponses" that the import resolves.

Example fix

# before (ImportError)
evaluator = MultiModalFaithfulnessEvaluator()

# after (option 1): pip install llama-index-llms-openai
evaluator = MultiModalFaithfulnessEvaluator()

# after (option 2): supply the LLM explicitly
evaluator = MultiModalFaithfulnessEvaluator(multi_modal_llm=my_openai_llm)

Defensive patterns

Strategy: try-catch

Validate before calling

try:
    from llama_index.llms.openai import OpenAIResponses  # noqa
    CAN_DEFAULT = True
except ImportError:
    CAN_DEFAULT = False

evaluator = (
    MultiModalFaithfulnessEvaluator()
    if CAN_DEFAULT
    else MultiModalFaithfulnessEvaluator(multi_modal_llm=my_llm)
)

Try / catch

try:
    evaluator = MultiModalFaithfulnessEvaluator()
except ImportError:
    raise SystemExit("install llama-index-llms-openai or pass multi_modal_llm=") from None

Prevention

Add llama-index-llms-openai (or the multi-modal extra) to your dependency list when using this evaluator.
Pass multi_modal_llm explicitly in production so behavior does not depend on an implicit fallback import.

When it happens

Trigger: Instantiating MultiModalFaithfulnessEvaluator() without multi_modal_llm in an environment lacking llama-index-llms-openai (e.g. only llama-index-core is installed).

Common situations: Slim installs of llama-index-core without integration extras; using a non-OpenAI multi-modal backend but forgetting to pass it explicitly; env corruption or wrong virtualenv; confusion caused by the misleading pip hint in the error text.

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/dc8af2a097dda5f6. Report an issue: GitHub.