mlflow/mlflow · error · ImportError

The `databricks-agents` package is required to use `mlflow.g

Error message

The `databricks-agents` package is required to use `mlflow.genai.judges.custom_prompt_judge` with model='databricks'. Please install it with `pip install databricks-agents`.

What it means

`custom_prompt_judge` with `model='databricks'` delegates to `databricks.agents.evals.judges.custom_prompt_judge`, which requires the `databricks-agents` package. When that import fails at call time, MLflow raises ImportError telling you to install it.

Source

Thrown at mlflow/genai/judges/custom_prompt_judge.py:82

    underscores, but should not contain spaces or special characters.

    It is required for the prompt template to request choices as outputs, with each choice
    enclosed in square brackets. Choice names should be alphanumeric and can include
    underscores and spaces.
    """
    model = model or get_default_model()

    if model == "databricks":
        try:
            from databricks.agents.evals.judges import custom_prompt_judge as db_custom_prompt_judge

            return db_custom_prompt_judge(
                name=name,
                prompt_template=prompt_template,
                numeric_values=numeric_values,
            )
        except ImportError:
            raise ImportError(
                "The `databricks-agents` package is required to use "
                "`mlflow.genai.judges.custom_prompt_judge` with model='databricks'. "
                "Please install it with `pip install databricks-agents`."
            )

    # Extract choices from the prompt template
    choices = _CHOICE_PATTERN.findall(prompt_template)

    if not choices:
        raise ValueError(
            "Prompt template must include choices denoted with [[CHOICE_NAME]]. "
            "No choices found in the provided prompt template."
        )

    # Validate that choices match numeric_values keys if provided
    if numeric_values is not None:
        sorted_numeric_values = sorted(numeric_values.keys())
        sorted_choices = sorted(choices)

View on GitHub (pinned to 6a27f2decc)

Solutions

1. Run `pip install databricks-agents`.
2. Add databricks-agents to your environment/requirements for deployments that use model='databricks'.
3. Alternatively pass an explicit LLM (e.g. a ChatCompletions-compatible model) so the databricks path is not used.
4. Switch to `make_judge` with a non-Databricks model if you want to avoid the Databricks dependency entirely.

Example fix

// before
judge = custom_prompt_judge(name="tone", prompt_template="...", model="databricks")

// after
# terminal
pip install databricks-agents
judge = custom_prompt_judge(name="tone", prompt_template="...", model="databricks")

import importlib.util
if model == "databricks" and importlib.util.find_spec("databricks.agents") is None:
    raise RuntimeError("model='databricks' requires: pip install databricks-agents")

try:
    judge = custom_prompt_judge(name=n, prompt_template=t, model="databricks")
except ImportError:
    judge = custom_prompt_judge(name=n, prompt_template=t, model=fallback_llm)

When it happens

Trigger: Calling `mlflow.genai.judges.custom_prompt_judge(name=..., prompt_template=..., model='databricks')` in an environment without `databricks-agents` installed.

Common situations: Using the Databricks-hosted model option outside a Databricks-managed environment; mlflow skinny installs; CI containers lacking the extra dependency; user forgot to pass a custom LLM (e.g. ChatCompletions endpoint) so it defaults to the databricks path.

Related errors

• The `databricks-agents` package is required to use `mlflow.g
• The `databricks-agents` package is required to use `mlflow.g
• Third-party scorer '{serialized.name}': could not import '{m
• Could not determine Kubernetes credentials. Ensure you are r
• EvaluationDataset is not available. It requires the mlflow.d

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/47a17eab912db33e. Report an issue: GitHub.