{"record":{"id":"efad5183209aa635","repo":"sgl-project/sglang","slug":"minimax-h3-ring-parallelism-requires-the-flashatte-efad51","errorCode":null,"errorMessage":"MiniMax H3 ring parallelism requires the FlashAttention backend (matches --ring-degree's general restriction).","messagePattern":"MiniMax H3 ring parallelism requires the FlashAttention backend \\(matches --ring-degree's general restriction\\)\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py","lineNumber":597,"sourceCode":"            _usp_input_all_to_all_packed_qkv,\n            _usp_output_all_to_all,\n        )\n\n        q, k, v = _usp_input_all_to_all_packed_qkv(q, k, v)\n\n    if attention._attention_impl is None:\n        attention._set_attention_backend(\n            get_attn_backend(\n                attention.head_dim,\n                q.dtype,\n                attention_requirements=AttentionRequirements(packed_varlen=True),\n            )\n        )\n\n    if ring_active:\n        ring_ws, _ = get_ring_ctx()\n        if attention._attention_backend_enum is not AttentionBackendEnum.FA:\n            raise NotImplementedError(\n                \"MiniMax H3 ring parallelism requires the FlashAttention \"\n                \"backend (matches --ring-degree's general restriction).\"\n            )\n        # max_seqlen is cu_seqlens[1] (`used`) by construction -- the real,\n        # non-padding row count ring needs, already a host int here.\n        out = _ring_attention_varlen(\n            q,\n            k,\n            v,\n            attn_impl=attention._attention_impl,\n            real_seq_len=max_seqlen,\n            ring_ws=ring_ws,\n        )\n    else:\n        if (\n            attention._attention_backend_enum\n            is AttentionBackendEnum.SUBBLOCK_SPARSE_ATTN\n        ):","sourceCodeStart":579,"sourceCodeEnd":615,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py#L579-L615","documentation":"MiniMax H3's attention core supports ring (sequence) parallelism only via the FlashAttention backend; when ring is active and the configured backend differs it raises NotImplementedError, mirroring the global --ring-degree restriction.","triggerScenarios":"Launching with --ring-degree > 1 (ring_active true) while the attention backend is set to something other than FA (e.g. FlashInfer, Triton, Torch) in server args or model runner config.","commonSituations":"Combining sequence parallelism with a backend chosen for other models on the same deployment; backend override flags or env vars; upgrading sglang where default backend changed away from FA.","solutions":["Set the attention backend to FA when using ring parallelism (e.g. --attention-backend fa / SGLANG_ATTN_BACKEND default) and keep --ring-degree","Drop ring parallelism (ring-degree 1 / remove the flag) if you must use the non-FA backend","Verify with get_ring_ctx() / server args at startup that ring and backend settings are consistent before serving"],"exampleFix":"# before\npython -m sglang.launch_server --model minimax-h3 --ring-degree 2 --attention-backend flashinfer\n\n# after\npython -m sglang.launch_server --model minimax-h3 --ring-degree 2 --attention-backend fa","handlingStrategy":"validation","validationCode":"from sglang.srt.layers.attention import AttentionBackendEnum\nassert attention_backend == AttentionBackendEnum.FA or ring_degree == 1, \\\n    'ring parallelism requires FA backend'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pair --ring-degree only with the FA backend","Fail fast at startup when ring_degree > 1 and backend != fa","Watch release notes for backend support changes"],"tags":["ring-parallelism","attention-backend","flashattention","minimax-h3","not-implemented"],"backgroundTag":"unsupported-backend-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}