{"record":{"id":"b1eaef9a83a17e06","repo":"sgl-project/sglang","slug":"external-ngram-corpus-exceeds-the-remaining-token","errorCode":null,"errorMessage":"External ngram corpus exceeds the remaining token budget ({max_tokens}) after loading {loaded_token_count} tokens.","messagePattern":"External ngram corpus exceeds the remaining token budget \\((.+?)\\) after loading (.+?) tokens\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/speculative/ngram_corpus.py","lineNumber":114,"sourceCode":"            return out_tokens.numpy().astype(np.int64), out_mask.numpy().astype(\n                np.int64\n            )\n\n        def erase_states(self, state_ids: List[int]) -> None:\n            state_ids_t = torch.tensor(state_ids, dtype=torch.int64)\n            self.erase_match_state(state_ids_t)  # type: ignore\n\n        def load_external_corpus_named(\n            self, corpus_id: str, chunks: Iterable[Sequence[int]], max_tokens: int\n        ) -> Tuple[int, int]:\n            self.start_external_corpus_load()  # type: ignore\n            chunk_count = 0\n            loaded_token_count = 0\n            try:\n                for chunk in chunks:\n                    tokens_t = torch.tensor(list(chunk), dtype=torch.int32)\n                    if loaded_token_count + len(tokens_t) > max_tokens:\n                        raise ValueError(\n                            \"External ngram corpus exceeds the remaining token budget \"\n                            f\"({max_tokens}) after loading {loaded_token_count} tokens.\"\n                        )\n                    loaded_token_count += len(tokens_t)\n                    self.append_external_corpus_tokens(tokens_t)  # type: ignore\n                    chunk_count += 1\n                self.finish_external_corpus_load(corpus_id)  # type: ignore\n            except Exception:\n                self.cancel_external_corpus_load()  # type: ignore\n                raise\n            return chunk_count, loaded_token_count\n\n        def remove_corpus(self, corpus_id: str) -> None:\n            self.remove_external_corpus(corpus_id)  # type: ignore\n\n        def list_corpora(self) -> Dict[str, int]:\n            result = self.list_external_corpora()  # type: ignore\n            if not result:","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/speculative/ngram_corpus.py#L96-L132","documentation":"Loading an external ngram corpus is capped by a token budget (external_corpus_max_tokens, minus already-loaded tokens). Exceeding the remaining budget mid-stream raises this error with the count loaded so far.","triggerScenarios":"Calling load_external_corpus_named with chunks whose cumulative token count exceeds external_corpus_max_tokens (default 10,000,000).","commonSituations":"Loading a large text/dataset as a draft corpus (e.g. multi-GB documents) with the default budget; repeated loads against the same named corpus without raising the limit.","solutions":["Increase external_corpus_max_tokens to cover the corpus size before loading.","Split/truncate the corpus to fit the remaining budget.","Track loaded_token_count and stop before exceeding the limit if partial corpora are acceptable."],"exampleFix":"// before\ncorpus = NgramCorpus(..., external_corpus_max_tokens=10_000_000)  # corpus is 50M tokens\n// after\ncorpus = NgramCorpus(..., external_corpus_max_tokens=100_000_000)","handlingStrategy":"validation","validationCode":"total = sum(len(list(c)) for c in chunks)\nif total > external_corpus_max_tokens - already_loaded:\n    raise/truncate before calling","typeGuard":null,"tryCatchPattern":"try:\n    corpus.load_external_corpus_named(name, chunks)\nexcept ValueError as e:\n    if 'token budget' in str(e): shrink corpus or raise limit","preventionTips":["Estimate corpus token count before loading and set external_corpus_max_tokens accordingly."],"tags":["resource-limit","ngram","corpus-loading"],"backgroundTag":"resource-budget-exceeded","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}