{"record":{"id":"9a5329eb6961a335","repo":"sgl-project/sglang","slug":"prefill-context-parallelism-with-the-trtllm-mha-pr","errorCode":null,"errorMessage":"Prefill context parallelism with the TRTLLM MHA prefill backend requires SM100 (trtllm-gen context kernel): the SM90/SM120 fmha_v2 prefill path does not implement CP shard masking.","messagePattern":"Prefill context parallelism with the TRTLLM MHA prefill backend requires SM100 \\(trtllm-gen context kernel\\): the SM90/SM120 fmha_v2 prefill path does not implement CP shard masking\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6588,"sourceCode":"                    > 0\n                )\n            ):\n                raise ValueError(\n                    \"TRTLLM FMHAv2 prefill on SM120 does not support \"\n                    \"fp8_e4m3 KV cache or skip-softmax.\"\n                )\n            if decode_backend == \"trtllm_mha\" and not (\n                is_sm90_supported() or is_sm100_supported() or is_sm120_supported()\n            ):\n                raise ValueError(\n                    \"TRTLLM MHA backend for decode is only supported on Hopper (SM90), Blackwell (SM100) and (SM120) GPUs. Please use a different decode backend.\"\n                )\n            if (\n                prefill_backend == \"trtllm_mha\"\n                and not is_sm100_supported()\n                and (cfg.enable_prefill_context_parallel or cfg.attn_cp_size > 1)\n            ):\n                raise ValueError(\n                    \"Prefill context parallelism with the TRTLLM MHA prefill backend \"\n                    \"requires SM100 (trtllm-gen context kernel): the SM90/SM120 \"\n                    \"fmha_v2 prefill path does not implement CP shard masking.\"\n                )\n\n        run_post_process_pass(self, _attention_backend_fa3_fp8_fallback)\n\n        run_post_process_pass(self, _fa4_page_constraint)\n\n        # AMD platforms backends\n        if resolved_view(self).attention_backend == \"aiter\":\n            if model_config.context_len > 8192:\n                self._declare(\n                    \"_handle_attention_backend_compatibility\",\n                    mem_fraction_static=cfg.mem_fraction_static * 0.85,\n                )\n\n        # Other platforms backends","sourceCodeStart":6570,"sourceCodeEnd":6606,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6570-L6606","documentation":"When prefill context parallelism (CP) is enabled with the trtllm_mha prefill backend, SGLang relies on the SM100 trtllm-gen context kernel, which implements CP shard masking. The SM90/SM120 fmha_v2 prefill path does not mask CP shards, so combining CP (enable_prefill_context_parallel or attn_cp_size > 1) with trtllm_mha prefill on non-SM100 GPUs is rejected at startup.","triggerScenarios":"Setting --enable-prefill-context-parallel or attn_cp_size > 1 with prefill backend trtllm_mha on a GPU where is_sm100_supported() is false (SM90 Hopper or SM120).","commonSituations":"Long-context prefill sharding attempts on H100 clusters; fleet configs enabling CP globally while also using trtllm_mha prefill; adopting attn_cp_size without checking backend support.","solutions":["Run on SM100 (Blackwell B200-class) hardware where the trtllm-gen context kernel implements CP masking","Use a different prefill backend (e.g. fa3 on Hopper) when CP is required","Disable prefill context parallelism (attn_cp_size=1, no --enable-prefill-context-parallel) with trtllm_mha prefill"],"exampleFix":"# before\npython -m sglang.launch_server --model M --prefill-attention-backend trtllm_mha --enable-prefill-context-parallel\n# after (on Hopper)\npython -m sglang.launch_server --model M --prefill-attention-backend fa3 --enable-prefill-context-parallel","handlingStrategy":"validation","validationCode":"import torch\ncap = torch.cuda.get_device_capability(0)\nsm = cap[0] * 10 + cap[1]\nuses_cp = args.enable_prefill_context_parallel or (args.attn_cp_size or 1) > 1\nif uses_cp and args.prefill_attention_backend == \"trtllm_mha\" and sm != 100:\n    args.prefill_attention_backend = \"fa3\" if sm == 90 else \"triton\"","typeGuard":null,"tryCatchPattern":"try:\n    ServerArgs(**kwargs)\nexcept ValueError as e:\n    if \"CP shard masking\" in str(e):\n        kwargs[\"prefill_attention_backend\"] = \"fa3\"\n        ServerArgs(**kwargs)\n    else:\n        raise","preventionTips":["Only pair trtllm_mha prefill CP with SM100 nodes in cluster configs","Check attn_cp_size/enable_prefill_context_parallel jointly with the backend in preflight","Keep CP-enabled and CP-free deployment profiles separate"],"tags":["sglang","context-parallel","attention-backend","trtllm","sm100","prefill"],"backgroundTag":"attention-backend-parallelism-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}