mlflow/mlflow · error · MlflowException

When starting a scorer, provided sample rate must be a numbe

Error message

When starting a scorer, provided sample rate must be a number

What it means

Scorer.start() registers/starts a scorer with a sampling configuration. Before persisting, it validates that sampling_config.sample_rate is numeric (int or float). A non-numeric value (e.g., a string like "0.5" or None) raises this MlflowException.invalid_parameter_value. The check exists because sample_rate drives server-side trace sampling math.

Source

Thrown at mlflow/genai/scorers/base.py:998

                print(f"Scorer is evaluating {active_scorer.sample_rate * 100}% of traces")

                # Start scorer with filter to only evaluate specific traces
                filtered_scorer = scorer.start(
                    sampling_config=ScorerSamplingConfig(
                        sample_rate=1.0, filter_string="YOUR_FILTER_STRING"
                    )
                )
        """
        from mlflow.genai.scorers.registry import (
            DatabricksStore,
            _get_scorer_store,
        )

        self._check_can_be_registered()

        sample_rate = sampling_config.sample_rate
        if not isinstance(sample_rate, (int, float)):
            raise MlflowException.invalid_parameter_value(
                "When starting a scorer, provided sample rate must be a number"
            )
        if sample_rate <= 0:
            raise MlflowException.invalid_parameter_value(
                "When starting a scorer, provided sample rate must be greater than 0"
            )

        scorer_name = name or self.name
        store = _get_scorer_store()

        if isinstance(store, DatabricksStore):
            return store.update_registered_scorer(
                name=scorer_name,
                scorer=self,
                sample_rate=sample_rate,
                filter_string=sampling_config.filter_string,
                experiment_id=experiment_id,
            )

View on GitHub (pinned to 6a27f2decc)

Solutions

1. Pass sample_rate as an int or float, e.g. scorer.start(sample_rate=0.5) not "0.5".
2. Coerce before calling: float(cfg["sample_rate"]) with a try/except ValueError.
3. If sample_rate may be absent, set an explicit default numeric value (e.g., 1.0) instead of None.

Example fix

// before
scorer.start(sampling_config=ScorerSamplingConfig(sample_rate="0.5"))
// after
scorer.start(sampling_config=ScorerSamplingConfig(sample_rate=0.5))

def valid_sample_rate(rate) -> bool:
    return isinstance(rate, (int, float)) and not isinstance(rate, bool)

if not valid_sample_rate(cfg.get("sample_rate")):
    raise ValueError("sample_rate must be int/float before calling start()")

def is_sample_rate(v) -> bool:
    return isinstance(v, (int, float)) and not isinstance(v, bool)

try:
    scorer.start(sample_rate=rate)
except MlflowException as e:
    if "sample rate must be a number" in str(e):
        scorer.start(sample_rate=float(rate))
    else:
        raise

When it happens

Trigger: Calling scorer.start(...) or constructing/submitting a ScorerSamplingConfig(sample_rate="0.5") where sample_rate is a str, None, bool-decoded JSON string, Decimal, or other non-int/float type.

Common situations: Loading config from YAML/env vars where sample_rate arrives as a string; passing None when the field was never set; JSON deserialization producing strings; users confusing sample_rate with a percentage string.

Related errors

• When updating a scorer, provided sample rate must be a numbe
• Session-level scorers require traces with session IDs. The f
• Every item in the list must be a dictionary.
• Invalid type for parameter `data`. Expected a list of dictio
• The dataset is empty. Please provide a non-empty dataset.

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/8c13f3cff9c06332. Report an issue: GitHub.