mlflow/mlflow · error · ImportError

The `databricks-agents` package is required to use `mlflow.g

Error message

The `databricks-agents` package is required to use `mlflow.genai.judges.{func.__name__}`. Please install it with `pip install databricks-agents`.

What it means

MLflow's built-in judges in `mlflow.genai.judges` are thin wrappers that delegate to the `databricks.agents.evals.judges` implementation shipped in the `databricks-agents` package. The `wrapper` decorator checks for that import at call time and raises ImportError when the package is missing.

Source

Thrown at mlflow/genai/judges/builtin.py:57

        feedback: The Feedback object to convert.

    Returns:
        A new Feedback object with our CategoricalRating.
    """
    feedback.value = CategoricalRating(feedback.value) if feedback.value else feedback.value
    return feedback


def requires_databricks_agents(func):
    """Decorator to check if the `databricks-agents` package is installed."""

    @wraps(func)
    def wrapper(*args, **kwargs):
        try:
            import databricks.agents.evals.judges  # noqa: F401

        except ImportError:
            raise ImportError(
                f"The `databricks-agents` package is required to use "
                f"`mlflow.genai.judges.{func.__name__}`. "
                "Please install it with `pip install databricks-agents`."
            )

        return func(*args, **kwargs)

    return wrapper


@format_docstring(_MODEL_API_DOC)
def is_context_relevant(
    *,
    request: str,
    context: Any,
    name: str | None = None,
    model: str | None = None,
    extra_headers: dict[str, str] | None = None,

View on GitHub (pinned to 6a27f2decc)

Solutions

Run `pip install databricks-agents`.
Add databricks-agents to requirements/dependency files used by the deployment environment.
If you cannot install databricks-agents, use `custom_prompt_judge` or `make_judge` (OpenAI-compatible) judges that don't require the package.
Pin the version alongside mlflow (e.g. `pip install 'databricks-agents>=0.x'`) to match the judge API expected by your mlflow version.

Example fix

// before
from mlflow.genai.judges import RelevanceToQuery
fb = RelevanceToQuery(name="rel", ...)

// after
# terminal
pip install databricks-agents
from mlflow.genai.judges import RelevanceToQuery
fb = RelevanceToQuery(name="rel", ...)

Defensive patterns

Strategy: try-catch

Validate before calling

import importlib.util
if importlib.util.find_spec("databricks.agents.evals.judges") is None:
    raise RuntimeError("pip install databricks-agents required for builtin judges")

Try / catch

try:
    feedback = mlflow.genai.judges.RelevanceToQuery(...)
except ImportError:
    # fall back to a judge that needs no databricks-agents
    feedback = custom_prompt_judge(name="rel", prompt_template=tmpl, model=my_llm)(...)

Prevention

Add databricks-agents to requirements for any job using mlflow.genai.judges
Sanity-check imports at service startup, not first request
Prefer custom_prompt_judge/make_judge in non-Databricks environments

When it happens

Trigger: Calling any `mlflow.genai.judges.<builtin judge>` (e.g. RelevanceToQuery, Guidelines, IsContextGrounded) without `databricks-agents` installed in the environment.

Common situations: Running in a non-Databricks or slim environment (mlflow skinny) where databricks-agents was never installed; deploying to CI/production containers with only mlflow core deps; forgetting that builtin judges are Databricks-hosted.

Related errors

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/d56d2162c2d5721e. Report an issue: GitHub.