{"record":{"id":"b1df70b39b18bc7c","repo":"mlflow/mlflow","slug":"scorers-scorer-details-return-non-numerical-va","errorCode":null,"errorMessage":"Scorers [{scorer_details}] return non-numerical values that cannot be automatically aggregated. Please provide an `objective` function to aggregate these values into a single score for optimization.","messagePattern":"Scorers \\[(.+?)\\] return non-numerical values that cannot be automatically aggregated\\. Please provide an `objective` function to aggregate these values into a single score for optimization\\.","errorType":"exception","errorClass":"MlflowException","httpStatus":null,"severity":"error","filePath":"mlflow/genai/optimize/util.py","lineNumber":210,"sourceCode":"            numeric_value = _convert_to_numeric(score)\n            if numeric_value is not None:\n                numeric_scores[name] = numeric_value\n\n        if objective is not None:\n            return objective(scores), rationales, numeric_scores\n\n        # If all scores were convertible, use sum as default aggregation\n        if len(numeric_scores) == len(scores):\n            # We average the scores to get the score between 0 and 1.\n            aggregated = sum(numeric_scores.values()) / len(numeric_scores)\n            return aggregated, rationales, numeric_scores\n\n        # Otherwise, report error with actual types\n        non_convertible = {\n            k: type(v).__name__ for k, v in scores.items() if k not in numeric_scores\n        }\n        scorer_details = \", \".join([f\"{k} (type: {t})\" for k, t in non_convertible.items()])\n        raise MlflowException(\n            f\"Scorers [{scorer_details}] return non-numerical values that cannot be \"\n            \"automatically aggregated. Please provide an `objective` function to aggregate \"\n            \"these values into a single score for optimization.\"\n        )\n\n    return metric\n","sourceCodeStart":192,"sourceCodeEnd":217,"githubUrl":"https://github.com/mlflow/mlflow/blob/6a27f2decc0b76eb1b54af31849784addb357dbc/mlflow/genai/optimize/util.py#L192-L217","documentation":"Raised when converting scorers to legacy metrics for optimization: the scorer returned a non-numerical value (e.g. string, dict, bool of non-numeric type) that MLflow cannot automatically aggregate into a single optimization objective. The optimizer needs one number per scorer to maximize/minimize.","triggerScenarios":"Calling optimize_prompts with custom scorers (or objective-less setups) whose returned Feedback values are non-numeric strings/objects, so _convert_scorer_to_legacy_metric cannot coerce them.","commonSituations":"Custom scorers returning text verdicts like 'good'/'bad' or dicts; LLM judges configured to output strings; refactor changed a scorer from returning 0-1 floats to labels; forgetting to supply an `objective` function.","solutions":["Provide an `objective` function to optimize_prompts that maps the scorer outputs to a single numeric score","Change the scorer to return numeric values (e.g. 0.0/1.0 instead of 'yes'/'no')","If a Feedback wraps the value, ensure feedback.value is numeric rather than the raw string"],"exampleFix":"// before\noptimize_prompts(..., scorers=[verdict_scorer])\n// after\noptimize_prompts(..., scorers=[verdict_scorer], objective=lambda scores: scores.get(\"verdict_scorer\") == \"yes\")","handlingStrategy":"type-guard","validationCode":"from mlflow.entities import Feedback\n\ndef scorer_values_numeric(scorers, sample_inputs, predict_fn):\n    for s in scorers:\n        fb = s(inputs=sample_inputs, outputs=predict_fn(sample_inputs), expectations={})\n        val = fb.value if isinstance(fb, Feedback) else fb\n        if val is not None and not isinstance(val, (int, float)):\n            raise TypeError(f\"{s.name} returns non-numeric {type(val).__name__}\")","typeGuard":"def is_numeric_score(v) -> bool:\n    import numbers\n    return isinstance(v, numbers.Number) and not isinstance(v, bool) or isinstance(v, bool)","tryCatchPattern":"from mlflow.exceptions import MlflowException\ntry:\n    mlflow.genai.optimize_prompts(..., scorers=scorers)\nexcept MlflowException as e:\n    if \"non-numerical values\" in str(e):\n        # retry with an objective that coerces to numbers\n        mlflow.genai.optimize_prompts(..., scorers=scorers, objective=lambda s: float(s))\n    else:\n        raise","preventionTips":["Make custom scorers return int/float values","Supply an objective function whenever scorers emit labels or objects","Smoke-test scorers on one sample before running optimization","Keep LLM judge output formats numeric (e.g. 1-5 scores)"],"tags":["mlflow","genai","scorers","aggregation"],"backgroundTag":"non-numeric-score","analyzedSha":"6a27f2decc0b76eb1b54af31849784addb357dbc","analyzedAt":"2026-08-29T20:54:51.419Z","schemaVersion":2},"datasetVersion":"2026-08-29T22:17:34.462Z"}