{"record":{"id":"2e054181a5c2928f","repo":"sgl-project/sglang","slug":"kv-canary-speculative-num-draft-tokens-must-be-no","errorCode":null,"errorMessage":"kv-canary: speculative_num_draft_tokens must be non-negative, got {spec_num_draft_tokens}","messagePattern":"kv-canary: speculative_num_draft_tokens must be non-negative, got (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/kv_canary/capacities.py","lineNumber":80,"sourceCode":"        if pool_slot_count <= 0:\n            raise ValueError(\n                f\"kv-canary: pool_slot_count must be positive, got {pool_slot_count}\"\n            )\n\n        cuda_graph_config = get_exec().graph.cuda_graph_config\n        cuda_graph_max_bs = (\n            cuda_graph_config.decode.max_bs if cuda_graph_config is not None else 0\n        ) or 0\n        if cuda_graph_max_bs < 0:\n            raise ValueError(\n                f\"kv-canary: cuda_graph_max_bs must be non-negative, got {cuda_graph_max_bs}\"\n            )\n\n        spec_num_draft_tokens = get_spec().speculative_num_draft_tokens\n        if spec_num_draft_tokens is None:\n            spec_num_draft_tokens = 0\n        if spec_num_draft_tokens < 0:\n            raise ValueError(\n                \"kv-canary: speculative_num_draft_tokens must be non-negative, \"\n                f\"got {spec_num_draft_tokens}\"\n            )\n\n        max_prefill_tokens = get_schedule().max_prefill_tokens\n        if max_prefill_tokens <= 0:\n            raise ValueError(\n                f\"kv-canary: max_prefill_tokens must be positive, got {max_prefill_tokens}\"\n            )\n\n        num_tokens_per_req = 1\n        if spec_num_draft_tokens:\n            num_tokens_per_req = max(num_tokens_per_req, spec_num_draft_tokens)\n\n        max_bs = max(cuda_graph_max_bs, req_to_token_pool_size)\n\n        chunked_prefill_size = get_schedule().chunked_prefill_size\n        chunked_limit = (","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/kv_canary/capacities.py#L62-L98","documentation":"from_args normalizes speculative_num_draft_tokens (None -> 0) from the speculative config and requires it to be non-negative, since draft-token count feeds per-forward canary capacities. A negative draft-token count is nonsensical and raises ValueError.","triggerScenarios":"Calling install_canary/from_args when the speculative decoding config carries a negative speculative_num_draft_tokens, e.g. a custom spec algorithm config or server-args override with a negative sentinel value.","commonSituations":"Scripts building speculative configs programmatically that use -1 as 'unset' instead of None/0; version changes where the draft-token field moved and stale configs supply invalid values.","solutions":["Set speculative_num_draft_tokens to None or 0 when speculative decoding is unused","Validate spec config values are non-negative before launching with --kv-canary enabled","Regenerate the server args / spec config with the current sglang version instead of reusing stale files"],"exampleFix":"# before\nServerArgs(..., speculative_num_draft_tokens=-1, kv_canary='raise')\n\n# after\nServerArgs(..., speculative_num_draft_tokens=None, kv_canary='raise')","handlingStrategy":"validation","validationCode":"n = server_args.speculative_num_draft_tokens\nif n is not None and n < 0:\n    raise SystemExit(\"speculative_num_draft_tokens must be None/0 or positive\")\n# then proceed with install_canary","typeGuard":"def valid_draft_tokens(n) -> bool:\n    return n is None or (isinstance(n, int) and n >= 0)","tryCatchPattern":null,"preventionTips":["Use None (not -1) to mean 'no speculative decoding'","Validate spec configs before enabling kv-canary","Regenerate stale server-args files across version upgrades"],"tags":["kv-canary","validation","speculative-decoding","value-error"],"backgroundTag":"config-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}