run-llama/llama_index · error · ValueError

contexts and response must be provided

Error message

contexts and response must be provided

What it means

Thrown by MultiModalFaithfulnessEvaluator.evaluate (sync) when contexts or response is None. The evaluator joins contexts into a prompt string and queries the multi-modal LLM with the response plus image blocks, so both text inputs are mandatory (query is discarded).

Source

Thrown at llama-index-core/llama_index/core/evaluation/multi_modal/faithfulness.py:146

        if "eval_template" in prompts:
            self._eval_template = prompts["eval_template"]
        if "refine_template" in prompts:
            self._refine_template = prompts["refine_template"]

    def evaluate(
        self,
        query: Union[str, None] = None,
        response: Union[str, None] = None,
        contexts: Union[Sequence[str], None] = None,
        image_paths: Union[List[str], None] = None,
        image_urls: Union[List[str], None] = None,
        **kwargs: Any,
    ) -> EvaluationResult:
        """Evaluate whether the response is faithful to the multi-modal contexts."""
        del query  # Unused
        del kwargs  # Unused
        if contexts is None or response is None:
            raise ValueError("contexts and response must be provided")

        context_str = "\n\n".join(contexts)
        fmt_prompt = self._eval_template.format(
            context_str=context_str, query_str=response
        )

        image_nodes: List[Union[ImageBlock, TextBlock]] = []

        if image_paths:
            image_nodes.extend(
                [ImageBlock(path=Path(image_path)) for image_path in image_paths]
            )
        if image_urls:
            image_nodes.extend([ImageBlock(url=image_url) for image_url in image_urls])

        image_nodes.append(TextBlock(text=fmt_prompt))

        response_obj = self._multi_modal_llm.chat(

View on GitHub (pinned to afd0fef371)

Solutions

1. Pass both text fields: evaluator.evaluate(response=str(r), contexts=["..."], image_paths=[...]).
2. If a document is image-only, still supply a minimal non-None contexts entry (e.g. a caption or placeholder string) plus the response.
3. Validate inputs before calling: skip or log items lacking response or contexts.

Example fix

# before
result = evaluator.evaluate(response=r, image_paths=paths)  # contexts missing

# after
result = evaluator.evaluate(
    response=str(r), contexts=texts or [""], image_paths=paths
)

if response is None or contexts is None:
    raise ValueError("multi-modal faithfulness needs response and text contexts")
result = evaluator.evaluate(response=str(response), contexts=list(contexts), image_paths=paths)

def mm_evaluable(response, contexts) -> bool:
    return response is not None and contexts is not None

When it happens

Trigger: Calling evaluator.evaluate(response=r, contexts=None) or evaluate(response=None, contexts=ctxs, image_paths=[...]); commonly when image paths/URLs are supplied but the text contexts were forgotten.

Common situations: Mixing up the signature — passing only image_paths/image_urls and omitting contexts; response objects that stringify to None after a failed generation; multimodal pipelines where text context extraction was skipped for image-only documents.

Related errors

• contexts and response must be provided
• The response is invalid
• code_execute_fn must be provided for CodeActAgent
• Unhandled shape {array.shape}.
• `llama-index-embeddings-clip` package not found, please run

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/6ef8044832b9da39. Report an issue: GitHub.