{"record":{"id":"643de53eda3f16d2","repo":"sgl-project/sglang","slug":"unsupported-profile-stage-forward-mode","errorCode":null,"errorMessage":"unsupported profile stage: {forward_mode=}","messagePattern":"unsupported profile stage: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/profile_utils.py","lineNumber":185,"sourceCode":"        logger.info(\n            f\"Profiling done. Traces are saved to: {self.profiler_kwargs['output_dir']}\"\n        )\n        self.profiler = None\n        # Clear the detailed step-span toggle here too: the v2 trigger auto-stop\n        # goes through _do_stop (not SchedulerProfilerManager._stop_profile), so\n        # this guarantees the flag resets on every stop path.\n        set_detailed_annotations_enabled(False)\n\n\ndef _get_stage_from_forward_mode(forward_mode: ForwardMode):\n    if forward_mode.is_prefill():\n        return \"prefill\"\n    elif forward_mode.is_decode():\n        return \"decode\"\n    elif forward_mode.is_idle():\n        return None\n    else:\n        raise RuntimeError(f\"unsupported profile stage: {forward_mode=}\")\n\n\n# ======================================== Stage related ==========================================\n\n\nclass _StageBasedTrigger:\n    @dataclass\n    class _StageConfig:\n        target_count: int\n\n    @dataclass\n    class _RunningState:\n        curr_stage: str\n        curr_count: int\n\n    def __init__(self, on_start: Callable, on_stop: Callable):\n        self.on_start = on_start\n        self.on_stop = on_stop","sourceCodeStart":167,"sourceCodeEnd":203,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/profile_utils.py#L167-L203","documentation":"While converting a batch's forward_mode into a profile stage, the mode was neither prefill, decode, nor idle, so the stage-triggered profiler cannot classify it and raises RuntimeError. This is an internal invariant violation, typically after a new ForwardMode was added without updating the profiler.","triggerScenarios":"Scheduler step() handling a forward_mode like MIXED, EXTEND, SPLIT_PREFILL (whatever is not covered by is_prefill/is_decode/is_idle) reaching _get_stage_from_forward_mode.","commonSituations":"Version mismatch: running a scheduler producing new forward modes with older profiler code, or custom attention/pipeline paths emitting unusual modes during profiling.","solutions":["Update sglang to a version where the profiler covers all forward modes","Disable stage-based profiling (--profile-stages unset) and use whole-run profiling","If developing a new forward mode, add a branch in _get_stage_from_forward_mode"],"exampleFix":"# before\nstage = _get_stage_from_forward_mode(forward_mode)  # MIXED -> RuntimeError\n# after\nif forward_mode.is_mixed():\n    stage = 'prefill'\nelse:\n    stage = _get_stage_from_forward_mode(forward_mode)","handlingStrategy":"fallback","validationCode":"null","typeGuard":"def is_supported_profile_mode(fm) -> bool:\n    return fm is not None and (fm.is_prefill() or fm.is_decode() or fm.is_idle())","tryCatchPattern":"try:\n    stage = _get_stage_from_forward_mode(forward_mode)\nexcept RuntimeError:\n    stage = None  # skip classifying this step","preventionTips":["Update sglang alongside scheduler/forward-mode changes","Run stage-based profiling in CI smoke tests to catch unclassified modes","Disable --profile-stages if an exotic mode appears"],"tags":["profiling","forward-mode","scheduler","internal-error","sglang"],"backgroundTag":"unhandled-enum-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}