{"record":{"id":"fee0bdae4b2d3400","repo":"sgl-project/sglang","slug":"trtllm-mha-backend-only-supports-topk-1-for-spec","errorCode":null,"errorMessage":"trtllm_mha backend only supports topk = 1 for speculative decoding.","messagePattern":"trtllm_mha backend only supports topk = 1 for speculative decoding\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/speculative_hook.py","lineNumber":763,"sourceCode":"        assert (\n            cfg.speculative_eagle_topk is None\n            and cfg.speculative_num_draft_tokens is None\n        )\n\n        steps, topk, draft_tokens = _auto_choose_speculative_params(\n            server_args, model_arch\n        )\n        declare_resolution(\n            server_args,\n            \"_handle_eagle_family.auto_params\",\n            speculative_num_steps=steps,\n            speculative_eagle_topk=topk,\n            speculative_num_draft_tokens=draft_tokens,\n        )\n\n    if \"trtllm_mha\" in attention_backends_of(resolved_view(server_args)):\n        if cfg.speculative_eagle_topk > 1:\n            raise ValueError(\n                \"trtllm_mha backend only supports topk = 1 for speculative decoding.\"\n            )\n\n    if cfg.speculative_use_rejection_sampling:\n        # Resolved alias by now: NEXTN -> EAGLE, Gemma4 draft -> FROZEN_KV_MTP.\n        # Only the EAGLE/EAGLE3 draft workers emit a target-vocab proposal that\n        # the rejection-sampling kernel consumes; everything else (STANDALONE,\n        # FROZEN_KV_MTP, NGRAM, DFLASH) is unsupported.\n        if cfg.speculative_algorithm not in (\"EAGLE\", \"EAGLE3\"):\n            raise NotImplementedError(\n                \"--speculative-use-rejection-sampling is only supported for \"\n                \"EAGLE / EAGLE3 / NEXTN, not \"\n                f\"speculative_algorithm={cfg.speculative_algorithm}.\"\n            )\n        if cfg.speculative_eagle_topk != 1:\n            raise ValueError(\n                \"--speculative-use-rejection-sampling requires --speculative-eagle-topk=1.\"\n            )","sourceCodeStart":745,"sourceCodeEnd":781,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/speculative_hook.py#L745-L781","documentation":"The trtllm_mha attention backend only supports EAGLE-family speculation with topk = 1 (linear single-chain drafting). Multi-branch drafting with speculative_eagle_topk > 1 is incompatible with that kernel's verify path.","triggerScenarios":"Selecting the trtllm_mha attention backend (directly or via --attention-backend) together with --speculative-eagle-topk > 1 on an EAGLE-family run.","commonSituations":"Performance-tuned launch configs that raise topk/steps for EAGLE3 while the backend is pinned to trtllm_mha (e.g. on Blackwell/TRT-LLM kernels).","solutions":["Set --speculative-eagle-topk 1 when using trtllm_mha","Switch attention backend (e.g. flashinfer/fa3) if topk > 1 is desired","Increase --speculative-num-steps instead of topk to raise speculation depth"],"exampleFix":"# before\n--attention-backend trtllm_mha --speculative-eagle-topk 4\n# after\n--attention-backend trtllm_mha --speculative-eagle-topk 1 --speculative-num-steps 4","handlingStrategy":"validation","validationCode":"if 'trtllm_mha' in (args.attention_backend or '') and (args.speculative_eagle_topk or 1) > 1:\n    raise SystemExit('trtllm_mha only supports speculative topk = 1')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["When pinning trtllm_mha, scale speculation via num_steps, not topk"],"tags":["speculative-decoding","attention-backend","trtllm-mha","eagle","topk"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}