{"record":{"id":"6d41a61a7aabef7c","repo":"sgl-project/sglang","slug":"batch-draft-token-num-config-value-config-must","errorCode":null,"errorMessage":"batch_draft_token_num config value ${config} must be less than or equal to draft_token_num: ${param_.draft_token_num}","messagePattern":"batch_draft_token_num config value (.+?) must be less than or equal to draft_token_num: (.+?)","errorType":"exception","errorClass":"std::runtime_error","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/jit/csrc/ngram_corpus/ngram.cpp","lineNumber":34,"sourceCode":"  }\n  if (!(param_.min_bfs_breadth > 0)) {\n    throw std::runtime_error(\n        \"min_bfs_breadth must be greater than 0, current value: \" + std::to_string(param_.min_bfs_breadth));\n  }\n  if (!(param_.min_bfs_breadth <= param_.max_bfs_breadth)) {\n    throw std::runtime_error(\n        \"min_bfs_breadth must be less than or equal to max_bfs_breadth, \"\n        \"current min_bfs_breadth: \" +\n        std::to_string(param_.min_bfs_breadth) + \", max_bfs_breadth: \" + std::to_string(param_.max_bfs_breadth));\n  }\n  if (!(param_.draft_token_num > 0)) {\n    throw std::runtime_error(\n        \"draft_token_num must be greater than 0, current value: \" + std::to_string(param_.draft_token_num));\n  }\n  for (auto config : param_.batch_draft_token_num) {\n    if (config != std::numeric_limits<decltype(config)>::max()) {\n      if (!(config <= param_.draft_token_num)) {\n        throw std::runtime_error(\n            \"batch_draft_token_num config value \" + std::to_string(config) +\n            \" must be less than or equal to draft_token_num: \" + std::to_string(param_.draft_token_num));\n      }\n    }\n  }\n\n  trie_ = std::make_unique<Trie>(capacity, param_);\n\n  insert_worker_ = std::thread(&Ngram::insertWorker, this);\n}\n\nNgram::~Ngram() {\n  insert_queue_.close();\n  if (insert_worker_.joinable()) {\n    insert_worker_.join();\n  }\n}\n","sourceCodeStart":16,"sourceCodeEnd":52,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/jit/csrc/ngram_corpus/ngram.cpp#L16-L52","documentation":"The Ngram constructor validates that every per-request value in batch_draft_token_num does not exceed the global draft_token_num. Each batch config controls how many draft tokens a single request may use, which can never exceed the engine-wide draft_token_num budget.","triggerScenarios":"Constructing ngram::Ngram with a Param where some batch_draft_token_num[i] (other than the sentinel numeric_limits::max()) is greater than draft_token_num.","commonSituations":"Setting per-request speculative draft lengths (e.g. [8, 64]) while leaving --speculative-num-draft-tokens at a smaller default; tuning batch draft sizes for mixed workloads without raising the global cap.","solutions":["Raise draft_token_num to at least max(batch_draft_token_num) (excluding sentinel max())","Lower the offending batch_draft_token_num entries to <= draft_token_num","Use the sentinel value (numeric_limits<int>::max(), i.e. -1/INT_MAX as configured) for entries that should default to the global limit"],"exampleFix":"// before\nparam.draft_token_num = 4;\nparam.batch_draft_token_num = {4, 16};\n// after\nparam.draft_token_num = 16;\nparam.batch_draft_token_num = {4, 16};","handlingStrategy":"validation","validationCode":"int dt = param.draft_token_num;\nfor (int v : param.batch_draft_token_num)\n  if (v != std::numeric_limits<int>::max() && v > dt)\n    throw std::invalid_argument(\"batch_draft_token_num entry too large\");","typeGuard":"bool params_consistent(const ngram::Param& p) {\n  int dt = p.draft_token_num;\n  if (dt <= 0) return false;\n  for (int v : p.batch_draft_token_num)\n    if (v != std::numeric_limits<int>::max() && v > dt) return false;\n  return true;\n}","tryCatchPattern":"try { Ngram n(param); } catch (const std::runtime_error& e) { /* fix config, log e.what() */ }","preventionTips":["Validate Param in one place before constructing Ngram","Clamp batch_draft_token_num entries to draft_token_num at config load"],"tags":["config-validation","ngram","speculative-decoding","constructor"],"backgroundTag":"config-value-out-of-range","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}