mlflow/mlflow · error · MlflowException

INVALID_PARAMETER_VALUE

INVALID_PARAMETER_VALUE

Error message

The `feedback_value_type` argument does not support a Literal typewith non-primitive types, but got {type(value).__name__}. Literal values must be str, int, float, or bool.

What it means

make_judge validates feedback_value_type and only permits Literal types whose values are primitive PbValueType members (str, int, float, bool). Passing a Literal containing any other value type (e.g. None, a tuple, bytes, or a dict) fails this check with INVALID_PARAMETER_VALUE.

Source

Thrown at mlflow/genai/judges/make_judge.py:57

    # Check for basic PbValueType (float, int, str, bool)
    pb_value_types = get_args(PbValueType)
    if feedback_value_type in pb_value_types:
        return

    # Check for Optional[T] / T | None where T is a single primitive PbValueType
    if _is_optional_pb_value_type(feedback_value_type, pb_value_types):
        return

    # Check for Literal type
    origin = get_origin(feedback_value_type)
    if origin is Literal:
        # Validate that all literal values are of PbValueType
        literal_values = get_args(feedback_value_type)
        for value in literal_values:
            if not isinstance(value, pb_value_types):
                from mlflow.exceptions import MlflowException

                raise MlflowException.invalid_parameter_value(
                    "The `feedback_value_type` argument does not support a Literal type"
                    f"with non-primitive types, but got {type(value).__name__}. "
                    f"Literal values must be str, int, float, or bool."
                )
        return

    # Check for dict[str, PbValueType]
    if origin is dict:
        args = get_args(feedback_value_type)
        if len(args) == 2:
            key_type, value_type = args
            # Key must be str
            if key_type != str:
                from mlflow.exceptions import MlflowException

                raise MlflowException.invalid_parameter_value(
                    f"dict key type must be str, got {key_type}"
                )

View on GitHub (pinned to 6a27f2decc)

Solutions

Remove non-primitive values from the Literal, keeping only str/int/float/bool members.
For an optional score, wrap instead of embedding None: Optional[Literal["yes", "no"]] or Literal["yes", "no"] | None.
Validate the Literal args before calling make_judge (see defense) or use plain str feedback_value_type if arbitrary values are acceptable.

Example fix

// before
make_judge(name="j", instructions="...", feedback_value_type=Literal["yes", "no", None])
// after
make_judge(name="j", instructions="...", feedback_value_type=Optional[Literal["yes", "no"]])

Defensive patterns

Strategy: type-guard

Validate before calling

from typing import Literal, get_args, get_origin
def check_literal(t):
    if get_origin(t) is Literal:
        bad = [v for v in get_args(t) if not isinstance(v, (str, int, float, bool))]
        if bad:
            raise ValueError(f"Literal has non-primitive values: {bad}")
    return t

check_literal(feedback_value_type)

Type guard

def is_primitive_literal(t) -> bool:
    from typing import Literal, get_args, get_origin
    return get_origin(t) is Literal and all(
        isinstance(v, (str, int, float, bool)) for v in get_args(t)
    )

Try / catch

from mlflow.exceptions import MlflowException
try:
    judge = make_judge(name="j", instructions="...", feedback_value_type=fvt)
except MlflowException as e:
    if "does not support a Literal type" in str(e):
        judge = make_judge(name="j", instructions="...", feedback_value_type=str)
    else:
        raise

Prevention

Use Optional[Literal[...]] instead of embedding None in a Literal.
Keep Literal members to str/int/float/bool only.
Check FeedbackValueType docs before choosing an exotic annotation.

When it happens

Trigger: Calling make_judge(feedback_value_type=Literal["yes", "no", None]) or Literal[(1,2)] or Literal[b"a"] — any Literal arg that is not str/int/float/bool.

Common situations: Trying to encode an optional categorical score as Literal[..., None] instead of Optional[Literal[...]]; accidentally passing tuple/list args; copying a Literal from an enum of non-primitive members.

Related errors

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/79b4ca33185c4b71. Report an issue: GitHub.