run-llama/llama_index · error · ImportError

ArgillaCallbackHandler is not installed. Please install it u

Error message

ArgillaCallbackHandler is not installed. Please install it using `pip install llama-index-callbacks-argilla`

What it means

Raised by create_global_handler() when eval_mode is 'argilla' but llama-index-callbacks-argilla is absent. Argilla feedback/tracing is an optional integration; the lazy import failure is caught and re-raised as an ImportError with the install command.

Source

Thrown at llama-index-core/llama_index/core/callbacks/global_handlers.py:97

        try:
            from llama_index.callbacks.deepeval import (
                deepeval_callback_handler,
            )  # pants: no-infer-dep
        except ImportError:
            raise ImportError(
                "DeepEvalCallbackHandler is not installed. "
                "Please install it using `pip install llama-index-callbacks-deepeval`"
            )
        handler = deepeval_callback_handler(**eval_params)
    elif eval_mode == "simple":
        handler = SimpleLLMHandler(**eval_params)
    elif eval_mode == "argilla":
        try:
            from llama_index.callbacks.argilla import (
                argilla_callback_handler,
            )  # pants: no-infer-dep
        except ImportError:
            raise ImportError(
                "ArgillaCallbackHandler is not installed. "
                "Please install it using `pip install llama-index-callbacks-argilla`"
            )
        handler = argilla_callback_handler(**eval_params)
    elif eval_mode == "langfuse":
        try:
            from llama_index.callbacks.langfuse import (
                langfuse_callback_handler,
            )  # pants: no-infer-dep
        except ImportError:
            raise ImportError(
                "LangfuseCallbackHandler is not installed. "
                "Please install it using `pip install llama-index-callbacks-langfuse`"
            )
        handler = langfuse_callback_handler(**eval_params)
    elif eval_mode == "agentops":
        try:
            from llama_index.callbacks.agentops import (

View on GitHub (pinned to afd0fef371)

Solutions

pip install llama-index-callbacks-argilla
Verify: python -c "from llama_index.callbacks.argilla import argilla_callback_handler"
Configure Argilla dataset/credentials before enabling the handler
Remove the argilla eval mode if tracing to Argilla was not intended

Example fix

# before
set_global_handler('argilla')  # ImportError

# after
# pip install llama-index-callbacks-argilla
set_global_handler('argilla')

Defensive patterns

Strategy: validation

Validate before calling

import importlib.util
if importlib.util.find_spec('llama_index.callbacks.argilla') is None:
    raise SystemExit('pip install llama-index-callbacks-argilla')

Try / catch

try:
    handler = create_global_handler('argilla')
except ImportError as e:
    handler = None

Prevention

Verify Argilla server reachability and the integration wheel together in a health check
Keep annotation pipelines' dependency sets explicit

When it happens

Trigger: create_global_handler('argilla', ...) or set_global_handler('argilla') with the package missing; `from llama_index.callbacks.argilla import argilla_callback_handler` fails.

Common situations: Using Argilla annotation workflows with LlamaIndex without the bridge package; fresh environments cloned from repos whose requirements file omits optional extras.

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/b132bb3fdc25532b. Report an issue: GitHub.