sgl-project/sglang · error · ValueError

MiniMax-H3 ring parallelism requires the FlashAttention back

Error message

MiniMax-H3 ring parallelism requires the FlashAttention backend for the transformer

What it means

MiniMax-H3 ring parallelism (ring_degree > 1) depends on FlashAttention-specific kernel behavior for its sequence-parallel attention ring, so validate_server_args rejects any ring_degree > 1 combined with a non-FA transformer attention backend (or None resolved to non-FA).

Source

Thrown at python/sglang/multimodal_gen/configs/pipeline_configs/minimax_h3.py:257

                if server_args.residency_mode(component) != LAYERWISE_OFFLOAD
            ]
            if missing_components:
                raise ValueError(
                    "MiniMax-H3 on MPS requires synchronous layerwise offload for "
                    f"{missing_components}; pass --layerwise-offload-components "
                    "transformer text_encoder video_vae audio_vae"
                )
            if server_args.enable_torch_compile:
                raise ValueError(
                    "MiniMax-H3 MPS execution does not support torch.compile; "
                    "pass --enable-torch-compile false"
                )
        selected_backend = self.resolve_transformer_attention_backend(server_args)
        if (
            int(server_args.ring_degree or 1) > 1
            and selected_backend is not AttentionBackendEnum.FA
        ):
            raise ValueError(
                "MiniMax-H3 ring parallelism requires the FlashAttention "
                "backend for the transformer"
            )
        if selected_backend is None:
            return
        get_attn_backend(
            self.dit_config.arch_config.attention_head_dim,
            torch.bfloat16,
            selected_attention_backend=selected_backend,
            attention_requirements=AttentionRequirements(packed_varlen=True),
        )

    def select_vae_weight_files(
        self,
        safetensors_list: list[str],
        component_model_path: str,
        component_name: str,
        vae_precision: str,

View on GitHub (pinned to 0132848349)

Solutions

1. Set the transformer attention backend to FA (e.g. --attention-backend fa) when using ring_degree > 1
2. Or drop ring parallelism (ring_degree=1) if FA is unavailable on your hardware
3. Use resolve_transformer_attention_backend(server_args) in your launch script/log to confirm what backend actually resolves before enabling ring degree

Example fix

# before
python -m sglang.launch_server --model MiniMax-H3 --ring-degree 2 --attention-backend flashinfer

# after
python -m sglang.launch_server --model MiniMax-H3 --ring-degree 2 --attention-backend fa

if int(server_args.ring_degree or 1) > 1:
    backend = config.resolve_transformer_attention_backend(server_args)
    assert backend is AttentionBackendEnum.FA, f"ring_degree>1 needs FA, got {backend}"

When it happens

Trigger: Launching with --ring-degree N (N>1) while the transformer attention backend resolves to something other than AttentionBackendEnum.FA — e.g. an explicit --attention-backend flashinfer/triton/sdpa, or a default that doesn't resolve to FA on this platform.

Common situations: Users enabling ring parallelism for long-context video while keeping a previously tuned non-FA backend; platform defaults (e.g. MPS or older GPUs) where FA is unavailable; config drift after upgrading attention backend enums.

Related errors

• MiniMax H3 ring parallelism requires the FlashAttention back
• MiniMax-H3 on MPS requires synchronous layerwise offload for
• MiniMax-H3 MPS execution does not support torch.compile; pas
• MiniMax H3 AdaLN cache max_plan_width must be positive; set
• MiniMax H3 Qwen3-VL encoders smaller than 32B require --comp

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a2c2d77513314606. Report an issue: GitHub.