{"record":{"id":"427e2164b708e0a6","repo":"sgl-project/sglang","slug":"batchmatch-expects-state-ids-tokens-and-total-le","errorCode":null,"errorMessage":"batchMatch expects state_ids, tokens, and total_lens to match in size","messagePattern":"batchMatch expects state_ids, tokens, and total_lens to match in size","errorType":"exception","errorClass":"std::runtime_error","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/jit/csrc/ngram_corpus/ngram.cpp","lineNumber":149,"sourceCode":"  for (;;) {\n    std::vector<int32_t> data;\n    if (!insert_queue_.dequeue(data)) {\n      break;\n    }\n    std::unique_lock<std::mutex> lock(mutex_);\n    trie_->insert(data.data(), data.size());\n    --pending_count_;\n    lock.unlock();\n    sync_cv_.notify_all();\n  }\n}\n\nResult Ngram::batchMatch(\n    const std::vector<int64_t>& state_ids,\n    const std::vector<std::vector<int32_t>>& tokens,\n    const std::vector<size_t>& total_lens) {\n  if (state_ids.size() != tokens.size() || state_ids.size() != total_lens.size()) {\n    throw std::runtime_error(\"batchMatch expects state_ids, tokens, and total_lens to match in size\");\n  }\n\n  std::unique_lock<std::mutex> lock(mutex_);\n\n  using TrieResultBuildFn =\n      Result (Trie::*)(const int32_t*, size_t, int32_t, size_t, const Param&, MatchState&, size_t) const;\n  using SamResultBuildFn = Result (SuffixAutomaton::*)(const int32_t*, size_t, int32_t, size_t, const Param&) const;\n  TrieResultBuildFn trie_result_build_fn;\n  SamResultBuildFn sam_result_build_fn;\n  if (param_.match_type == \"BFS\") {\n    trie_result_build_fn = &Trie::buildRecency;\n    sam_result_build_fn = &SuffixAutomaton::buildRecency;\n  } else if (param_.match_type == \"PROB\") {\n    trie_result_build_fn = &Trie::buildFrequency;\n    sam_result_build_fn = &SuffixAutomaton::buildFrequency;\n  } else {\n    throw std::runtime_error(\"Unknown match_type: '\" + param_.match_type + \"'. Must be 'BFS' or 'PROB'.\");\n  }","sourceCodeStart":131,"sourceCodeEnd":167,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/jit/csrc/ngram_corpus/ngram.cpp#L131-L167","documentation":"batchMatch takes parallel arrays: one state id, one token-tail vector, and one total length per request. It throws at entry if the three containers differ in length, since indexing them together would be out-of-bounds.","triggerScenarios":"Calling Ngram::batchMatch(state_ids, tokens, total_lens) where state_ids.size() != tokens.size() or != total_lens.size() — e.g. batch got truncated on the Python/FFI side before reaching C++.","commonSituations":"Mismatched batching when building request lists across components; a filter applied to tokens but not total_lens; empty-vs-partial batch assembly in the FFI wrapper.","solutions":["Build all three arrays from a single request list in one loop so they cannot diverge","Add an assert/equality check on the Python side before calling batch_match","If filtering requests, filter the struct/record, then unzip to the three arrays"],"exampleFix":"// before\nauto res = ngram.batchMatch(ids, tails, lens); // lens stale\n// after\nassert(ids.size() == tails.size() && ids.size() == lens.size());\nauto res = ngram.batchMatch(ids, tails, lens);","handlingStrategy":"type-guard","validationCode":"assert(state_ids.size() == tokens.size() && state_ids.size() == total_lens.size());","typeGuard":"bool batch_valid(const std::vector<int64_t>& ids, const std::vector<std::vector<int32_t>>& toks, const std::vector<size_t>& lens) { return ids.size() == toks.size() && ids.size() == lens.size(); }","tryCatchPattern":null,"preventionTips":["Build the three arrays from one request list in a single loop","Zip/unzip from a struct per request instead of parallel arrays"],"tags":["ngram","batch-validation","argument-mismatch"],"backgroundTag":"argument-length-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}