{"record":{"id":"264b403c6c8072ac","repo":"mlflow/mlflow","slug":"alignment-is-not-supported-for-session-level-score","errorCode":null,"errorMessage":"Alignment is not supported for session-level scorers.","messagePattern":"Alignment is not supported for session-level scorers\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"mlflow/genai/judges/base.py","lineNumber":133,"sourceCode":"\n        Returns:\n            A new Judge instance that is better aligned with the input traces.\n\n        Raises:\n            NotImplementedError: If called on a session-level scorer. Alignment is currently\n                only supported for single-turn scorers.\n\n        Note on Logging:\n            By default, alignment optimization shows minimal progress information.\n            To see detailed optimization output, set the optimizer's logger to DEBUG::\n\n                import logging\n\n                # For MemAlign optimizer (default)\n                logging.getLogger(\"mlflow.genai.judges.optimizers.memalign\").setLevel(logging.DEBUG)\n        \"\"\"\n        if self.is_session_level_scorer:\n            raise NotImplementedError(\"Alignment is not supported for session-level scorers.\")\n\n        if optimizer is None:\n            optimizer = get_default_optimizer()\n        return optimizer.align(self, traces)\n","sourceCodeStart":115,"sourceCodeEnd":138,"githubUrl":"https://github.com/mlflow/mlflow/blob/6a27f2decc0b76eb1b54af31849784addb357dbc/mlflow/genai/judges/base.py#L115-L138","documentation":"Aligning a scorer (optimizing its instructions/criteria against trace feedback) is intentionally unsupported for session-level scorers — scorers that evaluate an entire multi-span session rather than individual traces. The `align` method of Scorers (mlflow/genai/judges/base.py) raises NotImplementedError as a guard because the alignment optimizers operate on per-trace expectations.","triggerScenarios":"Calling `scorer.align(traces)` (or `mlflow.genai.optimize(...)` on a session-level scorer) where the scorer was registered/constructed with session-level semantics (is_session_level_scorer is True), e.g. a judge created to evaluate a whole session via `expectations` spanning multiple traces.","commonSituations":"Users attach scorers to a session-level evaluation (e.g. multi-turn agent conversations registered with session-level aggregation) then attempt automated alignment; code reused across trace-level and session-level scorers calls align unconditionally.","solutions":["Use a trace-level scorer instead if you need alignment — move the judge down to per-trace evaluation.","Manually tune the session-level scorer's instructions/criteria instead of calling align.","Check `scorer.is_session_level_scorer` before calling align and branch accordingly.","Upgrade MLflow: if you believe session-level alignment should exist, verify your version's docs; the API may have changed."],"exampleFix":"// before\noptimized = session_scorer.align(traces)  # NotImplementedError\n\n// after\nif session_scorer.is_session_level_scorer:\n    # align only trace-level scorers\n    optimized = trace_level_scorer.align(traces)\nelse:\n    optimized = session_scorer.align(traces)","handlingStrategy":"validation","validationCode":"if getattr(scorer, \"is_session_level_scorer\", False):\n    raise ValueError(\"scorer is session-level; align() is unsupported\")\noptimized = scorer.align(traces)","typeGuard":"def is_alignable(scorer) -> bool:\n    return not getattr(scorer, \"is_session_level_scorer\", False)","tryCatchPattern":"try:\n    optimized = scorer.align(traces)\nexcept NotImplementedError:\n    logger.warning(\"%s is session-level; skipping alignment\", scorer.name)\n    optimized = scorer","preventionTips":["Check is_session_level_scorer before any align() call","Keep session-level scorers out of alignment pipelines","Document which scorers in your suite are session-level"],"tags":["genai","scorers","not-implemented","alignment"],"backgroundTag":"unsupported-operation","analyzedSha":"6a27f2decc0b76eb1b54af31849784addb357dbc","analyzedAt":"2026-08-29T20:54:51.419Z","schemaVersion":2},"datasetVersion":"2026-08-29T22:17:34.462Z"}