{"record":{"id":"df746c619774e978","repo":"sgl-project/sglang","slug":"stochastic-rounding-for-the-mamba-ssm-cache-with","errorCode":null,"errorMessage":"Stochastic rounding for the Mamba SSM cache with --mamba-backend triton requires SM100 with CUDA >= 12.8 because it uses the cvt.rs.f16x2.f32 PTX instruction. On H100/SM90, run with --mamba-backend flashinfer --mamba-ssm-dtype float16, or disable --enable-mamba-cache-stochastic-rounding.","messagePattern":"Stochastic rounding for the Mamba SSM cache with --mamba-backend triton requires SM100 with CUDA >= 12\\.8 because it uses the cvt\\.rs\\.f16x2\\.f32 PTX instruction\\. On H100/SM90, run with --mamba-backend flashinfer --mamba-ssm-dtype float16, or disable --enable-mamba-cache-stochastic-rounding\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6772,"sourceCode":"                f\"integer, got {cfg.mamba_max_states_per_path}.\"\n            )\n\n        if cfg.enable_mamba_cache_stochastic_rounding:\n            if cfg.mamba_ssm_dtype != \"float16\":\n                raise ValueError(\n                    \"Stochastic rounding for the Mamba SSM cache requires \"\n                    f\"--mamba-ssm-dtype float16, got {cfg.mamba_ssm_dtype!r}. \"\n                    \"Run with --mamba-ssm-dtype float16 or disable \"\n                    \"--enable-mamba-cache-stochastic-rounding.\"\n                )\n            if not is_cuda():\n                raise ValueError(\n                    \"Stochastic rounding for the Mamba SSM cache is only \"\n                    \"supported on NVIDIA CUDA platforms. Disable \"\n                    \"--enable-mamba-cache-stochastic-rounding on this platform.\"\n                )\n            if cfg.mamba_backend == \"triton\" and not is_sm100_supported():\n                raise ValueError(\n                    \"Stochastic rounding for the Mamba SSM cache with \"\n                    \"--mamba-backend triton requires SM100 with CUDA >= 12.8 \"\n                    \"because it uses the cvt.rs.f16x2.f32 PTX instruction. On \"\n                    \"H100/SM90, run with --mamba-backend flashinfer \"\n                    \"--mamba-ssm-dtype float16, or disable \"\n                    \"--enable-mamba-cache-stochastic-rounding.\"\n                )\n\n        if cfg.mamba_backend == \"flashinfer\":\n            flashinfer_error = (\n                \"FlashInfer mamba module not available, please check the \"\n                \"FlashInfer installation.\"\n            )\n            if cfg.enable_mamba_cache_stochastic_rounding:\n                flashinfer_error += (\n                    \" Stochastic rounding with --mamba-backend flashinfer \"\n                    \"requires FlashInfer Mamba and --mamba-ssm-dtype float16.\"\n                )","sourceCodeStart":6754,"sourceCodeEnd":6790,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6754-L6790","documentation":"The triton Mamba backend's stochastic rounding uses the cvt.rs.f16x2.f32 PTX instruction, which only exists on SM100 (Blackwell) with CUDA >= 12.8. On older GPUs (H100/SM90 etc.) the combination --mamba-backend triton + --enable-mamba-cache-stochastic-rounding is rejected.","triggerScenarios":"Launching with --enable-mamba-cache-stochastic-rounding --mamba-backend triton on a GPU where is_sm100_supported() is False (H100, A100, or CUDA < 12.8).","commonSituations":"A config tuned on a B200/Blackwell box reused on an H100 cluster; older CUDA toolkit (12.4) installed so is_sm100_supported() is False even on new hardware.","solutions":["On SM90: switch to --mamba-backend flashinfer --mamba-ssm-dtype float16 and keep stochastic rounding","Disable --enable-mamba-cache-stochastic-rounding and keep triton","Upgrade the CUDA toolkit to >= 12.8 if you are actually on SM100 hardware"],"exampleFix":"# before (on H100)\n--mamba-backend triton --enable-mamba-cache-stochastic-rounding --mamba-ssm-dtype float16\n# after (on H100)\n--mamba-backend flashinfer --enable-mamba-cache-stochastic-rounding --mamba-ssm-dtype float16","handlingStrategy":"validation","validationCode":"import torch\nfrom sglang.srt.utils import is_sm100_supported\nif args.enable_mamba_cache_stochastic_rounding and args.mamba_backend == \"triton\":\n    assert is_sm100_supported(), \"triton stochastic rounding needs SM100 + CUDA>=12.8\"","typeGuard":"null","tryCatchPattern":null,"preventionTips":["Check GPU architecture (SM90 vs SM100) before choosing mamba backend flags","Keep per-cluster GPU profiles: H100 uses flashinfer+float16, Blackwell may use triton"],"tags":["sglang","mamba","triton","sm100","gpu-architecture","stochastic-rounding"],"backgroundTag":"gpu-architecture-requirement","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}