{"record":{"id":"83dda4add75b0c7b","repo":"sgl-project/sglang","slug":"speculative-use-rejection-sampling-requires-sp","errorCode":null,"errorMessage":"--speculative-use-rejection-sampling requires --speculative-eagle-topk=1.","messagePattern":"--speculative-use-rejection-sampling requires --speculative-eagle-topk=1\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/speculative_hook.py","lineNumber":779,"sourceCode":"    if \"trtllm_mha\" in attention_backends_of(resolved_view(server_args)):\n        if cfg.speculative_eagle_topk > 1:\n            raise ValueError(\n                \"trtllm_mha backend only supports topk = 1 for speculative decoding.\"\n            )\n\n    if cfg.speculative_use_rejection_sampling:\n        # Resolved alias by now: NEXTN -> EAGLE, Gemma4 draft -> FROZEN_KV_MTP.\n        # Only the EAGLE/EAGLE3 draft workers emit a target-vocab proposal that\n        # the rejection-sampling kernel consumes; everything else (STANDALONE,\n        # FROZEN_KV_MTP, NGRAM, DFLASH) is unsupported.\n        if cfg.speculative_algorithm not in (\"EAGLE\", \"EAGLE3\"):\n            raise NotImplementedError(\n                \"--speculative-use-rejection-sampling is only supported for \"\n                \"EAGLE / EAGLE3 / NEXTN, not \"\n                f\"speculative_algorithm={cfg.speculative_algorithm}.\"\n            )\n        if cfg.speculative_eagle_topk != 1:\n            raise ValueError(\n                \"--speculative-use-rejection-sampling requires --speculative-eagle-topk=1.\"\n            )\n        if (\n            cfg.speculative_accept_threshold_single != 1.0\n            or cfg.speculative_accept_threshold_acc != 1.0\n        ):\n            raise ValueError(\n                \"--speculative-use-rejection-sampling is incompatible with \"\n                \"--speculative-accept-threshold-single / \"\n                \"--speculative-accept-threshold-acc; rejection sampling ignores \"\n                \"the accept thresholds.\"\n            )\n        if cfg.enable_deterministic_inference:\n            raise ValueError(\n                \"--speculative-use-rejection-sampling is incompatible with \"\n                \"--enable-deterministic-inference; the sampling kernel draws \"\n                \"coins from the global RNG and is not batch-invariant.\"\n            )","sourceCodeStart":761,"sourceCodeEnd":797,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/speculative_hook.py#L761-L797","documentation":"Rejection sampling in SGLang is only implemented for the linear (topk=1) draft chain: with a single draft hypothesis the verifier can do an exact accept/reject coin flip against the target distribution. With --speculative-eagle-topk > 1 there are multiple draft branches and the rejection-sampling kernel does not support that shape, so the args hook rejects the combination with a ValueError.","triggerScenarios":"Launching with --speculative-use-rejection-sampling and --speculative-eagle-topk set to a value other than 1 (e.g. 4 or 8 for tree-based drafting). The check fires during handle_server_args validation in _handle_eagle_family.","commonSituations":"Copying a tree-based EAGLE config (topk=4/8) from a benchmark and adding the rejection-sampling flag; tuning topk for throughput and forgetting the flag constrains it to 1.","solutions":["Set --speculative-eagle-topk 1 and keep --speculative-use-rejection-sampling","Or drop --speculative-use-rejection-sampling and keep tree-based topk > 1 (uses the standard verify path)","If you need both tree drafting and exact sampling, wait for/wrap a kernel that supports topk>1; not currently available"],"exampleFix":"# before\n--speculative-use-rejection-sampling --speculative-eagle-topk 4\n# after\n--speculative-use-rejection-sampling --speculative-eagle-topk 1","handlingStrategy":"validation","validationCode":"if server_args.speculative_use_rejection_sampling:\n    assert server_args.speculative_eagle_topk == 1, \"rejection sampling requires topk=1\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat topk=1 as part of the rejection-sampling preset, not an independent knob","Document tuned EAGLE configs with which verify path they target"],"tags":["speculative-decoding","rejection-sampling","eagle-topk","server-args"],"backgroundTag":"incompatible-launch-flags","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}