sgl-project/sglang · error · std::runtime_error

appendExternalCorpusTokens called without startExternalCorpu

Error message

appendExternalCorpusTokens called without startExternalCorpusLoad

What it means

appendExternalCorpusTokens is a staging API that only works between startExternalCorpusLoad and finishExternalCorpusLoad. It throws when staging_sam_ is null, i.e. no load was started or the previous one already completed/failed.

Source

Thrown at python/sglang/kernels/jit/csrc/ngram_corpus/ngram.cpp:81

  for (auto&& token : tokens) {
    insert_queue_.enqueue(std::move(token));
  }
}

// NOTE: staging operations (start/append/finish) are called from a background
// thread during async corpus loading. They do NOT hold mutex_ because
// staging_sam_ is disjoint from sams_ / trie_. Only finishExternalCorpusLoad
// briefly acquires mutex_ when moving the completed SAM into sams_.
void Ngram::startExternalCorpusLoad() {
  if (staging_sam_) {
    throw std::runtime_error("startExternalCorpusLoad called while another load is in progress");
  }
  staging_sam_ = std::make_unique<SuffixAutomaton>();
}

void Ngram::appendExternalCorpusTokens(const std::vector<int32_t>& tokens) {
  if (!staging_sam_) {
    throw std::runtime_error("appendExternalCorpusTokens called without startExternalCorpusLoad");
  }
  staging_sam_->appendTokens(tokens);
}

void Ngram::finishExternalCorpusLoad(const std::string& corpus_id) {
  if (!staging_sam_) {
    throw std::runtime_error("finishExternalCorpusLoad called without startExternalCorpusLoad");
  }
  staging_sam_->finalize();
  if (staging_sam_->empty()) {
    staging_sam_.reset();
    throw std::runtime_error("External corpus is empty — no tokens were loaded.");
  }
  // Only lock briefly to install the completed SAM.
  std::unique_lock<std::mutex> lock(mutex_);
  if (sams_.find(corpus_id) != sams_.end()) {
    throw std::runtime_error(
        "External corpus '" + corpus_id + "' already exists. Remove it before adding a new corpus with the same id.");

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure startExternalCorpusLoad() is called (and awaited) before any appendExternalCorpusTokens call
2. Add a state check / boolean flag in your loader before appending
3. Serialize the start/append/finish sequence on one thread or with a mutex

Example fix

// before
ngram.appendExternalCorpusTokens(chunk); // no start yet
// after
ngram.startExternalCorpusLoad();
ngram.appendExternalCorpusTokens(chunk);

bool load_started = false; // set true right after startExternalCorpusLoad()
if (load_started) ngram.appendExternalCorpusTokens(chunk);

try { ngram.appendExternalCorpusTokens(chunk); } catch (const std::runtime_error& e) { /* restart load: start() then append */ }

When it happens

Trigger: Calling appendExternalCorpusTokens(tokens) without a prior successful startExternalCorpusLoad(), or after finishExternalCorpusLoad() already consumed the staging automaton.

Common situations: Background loader thread racing ahead of the starter thread; retrying appends after a failed finish; reordered async pipeline where chunk streaming begins before start.

Related errors

• finishExternalCorpusLoad called without startExternalCorpusL
• Missing previous frame for delta payload
• startExternalCorpusLoad called while another load is in prog
• External ngram corpus exceeds the remaining token budget ({m
• External corpus is empty — no tokens were loaded.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/267f8260ef862aa5. Report an issue: GitHub.