{"record":{"id":"5d6f7698daddd1ec","repo":"sgl-project/sglang","slug":"speculative-use-rejection-sampling-with-multi-la","errorCode":null,"errorMessage":"--speculative-use-rejection-sampling with multi-layer EAGLE (--enable-multi-layer-eagle) requires --speculative-eagle-topk 1; rejection sampling is only implemented for the linear (topk=1) chain.","messagePattern":"--speculative-use-rejection-sampling with multi-layer EAGLE \\(--enable-multi-layer-eagle\\) requires --speculative-eagle-topk 1; rejection sampling is only implemented for the linear \\(topk=1\\) chain\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/speculative_hook.py","lineNumber":805,"sourceCode":"                \"--speculative-use-rejection-sampling is incompatible with \"\n                \"--speculative-accept-threshold-single / \"\n                \"--speculative-accept-threshold-acc; rejection sampling ignores \"\n                \"the accept thresholds.\"\n            )\n        if cfg.enable_deterministic_inference:\n            raise ValueError(\n                \"--speculative-use-rejection-sampling is incompatible with \"\n                \"--enable-deterministic-inference; the sampling kernel draws \"\n                \"coins from the global RNG and is not batch-invariant.\"\n            )\n\n        from sglang.srt.arg_groups.overrides import resolved_view\n\n        if (\n            resolved_view(server_args).enable_multi_layer_eagle\n            and cfg.speculative_eagle_topk != 1\n        ):\n            raise ValueError(\n                \"--speculative-use-rejection-sampling with multi-layer EAGLE \"\n                \"(--enable-multi-layer-eagle) requires --speculative-eagle-topk 1; \"\n                \"rejection sampling is only implemented for the linear (topk=1) chain.\"\n            )\n        logger.info(\n            \"Rejection sampling is enabled for speculative decoding \"\n            \"(speculative_use_rejection_sampling=True).\"\n        )\n\n    if (\n        cfg.speculative_eagle_topk == 1\n        and cfg.speculative_num_draft_tokens != cfg.speculative_num_steps + 1\n    ):\n        logger.warning(\n            \"speculative_num_draft_tokens is adjusted to speculative_num_steps + 1 when speculative_eagle_topk == 1\"\n        )\n        declare_resolution(\n            server_args,","sourceCodeStart":787,"sourceCodeEnd":823,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/speculative_hook.py#L787-L823","documentation":"Multi-layer EAGLE (draft chains with several verification layers) is only compatible with the rejection-sampling kernel when the draft tree is a linear chain, i.e. --speculative-eagle-topk=1. With topk>1 each layer branches into a tree and the rejection-sampling implementation cannot handle it. The check uses the resolved-view value of enable_multi_layer_eagle (set via model overrides), so even indirect enablement is caught.","triggerScenarios":"Launching with --speculative-use-rejection-sampling while enable_multi_layer_eagle resolves true (CLI flag or model-override) and --speculative-eagle-topk != 1. Fires after the earlier topk==1 check is bypassed via override layers.","commonSituations":"Using a model whose server-args overrides enable multi-layer EAGLE by default (e.g. via resolved_view overrides) combined with a copied EAGLE topk>1 config and rejection sampling; changing topk after a multi-layer setup was working.","solutions":["Set --speculative-eagle-topk 1 when using multi-layer EAGLE with rejection sampling","Or disable multi-layer EAGLE (remove --enable-multi-layer-eagle / the model override) and keep topk as-is","Or drop --speculative-use-rejection-sampling to use the tree verify path"],"exampleFix":"# before\n--speculative-use-rejection-sampling --enable-multi-layer-eagle --speculative-eagle-topk 4\n# after\n--speculative-use-rejection-sampling --enable-multi-layer-eagle --speculative-eagle-topk 1","handlingStrategy":"validation","validationCode":"from sglang.srt.arg_groups.overrides import resolved_view\nif server_args.speculative_use_rejection_sampling:\n    v = resolved_view(server_args)\n    assert not v.enable_multi_layer_eagle or server_args.speculative_eagle_topk == 1","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check the resolved view, not just CLI flags — model overrides can enable multi-layer EAGLE","Pin topk=1 for any rejection-sampling launch template"],"tags":["speculative-decoding","rejection-sampling","multi-layer-eagle","server-args"],"backgroundTag":"incompatible-launch-flags","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}