sgl-project/sglang · error · ValueError

External ngram corpus exceeds the remaining token budget ({m

Error message

External ngram corpus exceeds the remaining token budget ({max_tokens}) after loading {loaded_token_count} tokens.

What it means

Loading an external ngram corpus is capped by a token budget (external_corpus_max_tokens, minus already-loaded tokens). Exceeding the remaining budget mid-stream raises this error with the count loaded so far.

Source

Thrown at python/sglang/kernels/ops/speculative/ngram_corpus.py:114

            return out_tokens.numpy().astype(np.int64), out_mask.numpy().astype(
                np.int64
            )

        def erase_states(self, state_ids: List[int]) -> None:
            state_ids_t = torch.tensor(state_ids, dtype=torch.int64)
            self.erase_match_state(state_ids_t)  # type: ignore

        def load_external_corpus_named(
            self, corpus_id: str, chunks: Iterable[Sequence[int]], max_tokens: int
        ) -> Tuple[int, int]:
            self.start_external_corpus_load()  # type: ignore
            chunk_count = 0
            loaded_token_count = 0
            try:
                for chunk in chunks:
                    tokens_t = torch.tensor(list(chunk), dtype=torch.int32)
                    if loaded_token_count + len(tokens_t) > max_tokens:
                        raise ValueError(
                            "External ngram corpus exceeds the remaining token budget "
                            f"({max_tokens}) after loading {loaded_token_count} tokens."
                        )
                    loaded_token_count += len(tokens_t)
                    self.append_external_corpus_tokens(tokens_t)  # type: ignore
                    chunk_count += 1
                self.finish_external_corpus_load(corpus_id)  # type: ignore
            except Exception:
                self.cancel_external_corpus_load()  # type: ignore
                raise
            return chunk_count, loaded_token_count

        def remove_corpus(self, corpus_id: str) -> None:
            self.remove_external_corpus(corpus_id)  # type: ignore

        def list_corpora(self) -> Dict[str, int]:
            result = self.list_external_corpora()  # type: ignore
            if not result:

View on GitHub (pinned to 0132848349)

Solutions

1. Increase external_corpus_max_tokens to cover the corpus size before loading.
2. Split/truncate the corpus to fit the remaining budget.
3. Track loaded_token_count and stop before exceeding the limit if partial corpora are acceptable.

Example fix

// before
corpus = NgramCorpus(..., external_corpus_max_tokens=10_000_000)  # corpus is 50M tokens
// after
corpus = NgramCorpus(..., external_corpus_max_tokens=100_000_000)

total = sum(len(list(c)) for c in chunks)
if total > external_corpus_max_tokens - already_loaded:
    raise/truncate before calling

try:
    corpus.load_external_corpus_named(name, chunks)
except ValueError as e:
    if 'token budget' in str(e): shrink corpus or raise limit

When it happens

Trigger: Calling load_external_corpus_named with chunks whose cumulative token count exceeds external_corpus_max_tokens (default 10,000,000).

Common situations: Loading a large text/dataset as a draft corpus (e.g. multi-GB documents) with the default budget; repeated loads against the same named corpus without raising the limit.

Related errors

• startExternalCorpusLoad called while another load is in prog
• appendExternalCorpusTokens called without startExternalCorpu
• finishExternalCorpusLoad called without startExternalCorpusL
• External corpus is empty — no tokens were loaded.
• External corpus '${corpus_id}' already exists. Remove it bef

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b1eaef9a83a17e06. Report an issue: GitHub.