run-llama/llama_index · error · ValueError

Eval mode {eval_mode} not supported.

Error message

Eval mode {eval_mode} not supported.

What it means

Raised by create_global_handler() when the eval_mode string does not match any supported branch. Supported modes are wandb, openinference, arize_phoenix, honeyhive, promptlayer, deepeval, simple, argilla, langfuse, agentops, literalai, and opik; anything else (including typos and case mismatches) falls through to raise ValueError. Unlike the ImportError branches, this is purely a caller-input error, not a packaging issue.

Source

Thrown at llama-index-core/llama_index/core/callbacks/global_handlers.py:147

        except ImportError:
            raise ImportError(
                "Literal AI Handler is not installed. "
                "Please install it using `pip install llama-index-callbacks-literalai`"
            )
        handler = literalai_callback_handler(**eval_params)
    elif eval_mode == "opik":
        try:
            from llama_index.callbacks.opik import (
                opik_callback_handler,
            )  # pants: no-infer-dep
        except ImportError:
            raise ImportError(
                "Opik Handler is not installed. "
                "Please install it using `pip install llama-index-callbacks-opik`"
            )
        handler = opik_callback_handler(**eval_params)
    else:
        raise ValueError(f"Eval mode {eval_mode} not supported.")

    return handler

View on GitHub (pinned to afd0fef371)

Solutions

1. Use one of the exact supported strings: 'wandb', 'openinference', 'arize_phoenix', 'honeyhive', 'promptlayer', 'deepeval', 'simple', 'argilla', 'langfuse', 'agentops', 'literalai', 'opik'
2. Check for typos, casing, and stray whitespace in the eval_mode value (e.g. from env vars or YAML config)
3. Validate eval_mode against the supported list before calling, especially when it comes from user config
4. If you expected a mode that is not listed, your llama-index-core version may predate it — upgrade llama-index-core

Example fix

# before
from llama_index.core.callbacks.global_handlers import create_global_handler
handler = create_global_handler('phoenix')  # ValueError

# after
handler = create_global_handler('arize_phoenix')

SUPPORTED_MODES = {'wandb','openinference','arize_phoenix','honeyhive','promptlayer','deepeval','simple','argilla','langfuse','agentops','literalai','opik'}
assert eval_mode in SUPPORTED_MODES, f'eval_mode must be one of {sorted(SUPPORTED_MODES)}'

try:
    handler = create_global_handler(eval_mode)
except ValueError as e:
    raise ConfigError(f'Bad eval mode {eval_mode!r}: {e}') from e

When it happens

Trigger: create_global_handler('phoenix', ...), set_global_handler('WandB'), set_global_handler('open-inference'), or any misspelled/renamed mode; also modes removed or renamed between llama-index versions.

Common situations: Typos or wrong casing in config files/env vars that feed eval_mode; using an integration name (e.g. 'openinference') that is correct but its package name ('otel') is guessed instead; mode lists drifting across library upgrades.

Related errors

• Max iterations of {max_iterations} reached! Either something
• CitableBlock content must contain exactly one block when pro
• Cannot add two handlers of the same type {type(new_handler)}
• WandbCallbackHandler is not installed. Please install it usi
• OpenInferenceCallbackHandler is not installed. Please instal

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/5bce7bbd30bcfc8e. Report an issue: GitHub.