sgl-project/sglang · error · std::runtime_error

startExternalCorpusLoad called while another load is in prog

Error message

startExternalCorpusLoad called while another load is in progress

What it means

The external-corpus staging API is single-slot: only one background corpus load may be active at a time. startExternalCorpusLoad throws if a staging suffix automaton already exists, i.e. a previous start was not finished or abandoned.

Source

Thrown at python/sglang/kernels/jit/csrc/ngram_corpus/ngram.cpp:74

}

void Ngram::asyncInsert(std::vector<std::vector<int32_t>>&& tokens) {
  {
    std::lock_guard<std::mutex> lock(mutex_);
    pending_count_ += tokens.size();
  }
  for (auto&& token : tokens) {
    insert_queue_.enqueue(std::move(token));
  }
}

// NOTE: staging operations (start/append/finish) are called from a background
// thread during async corpus loading. They do NOT hold mutex_ because
// staging_sam_ is disjoint from sams_ / trie_. Only finishExternalCorpusLoad
// briefly acquires mutex_ when moving the completed SAM into sams_.
void Ngram::startExternalCorpusLoad() {
  if (staging_sam_) {
    throw std::runtime_error("startExternalCorpusLoad called while another load is in progress");
  }
  staging_sam_ = std::make_unique<SuffixAutomaton>();
}

void Ngram::appendExternalCorpusTokens(const std::vector<int32_t>& tokens) {
  if (!staging_sam_) {
    throw std::runtime_error("appendExternalCorpusTokens called without startExternalCorpusLoad");
  }
  staging_sam_->appendTokens(tokens);
}

void Ngram::finishExternalCorpusLoad(const std::string& corpus_id) {
  if (!staging_sam_) {
    throw std::runtime_error("finishExternalCorpusLoad called without startExternalCorpusLoad");
  }
  staging_sam_->finalize();
  if (staging_sam_->empty()) {
    staging_sam_.reset();

View on GitHub (pinned to 0132848349)

Solutions

1. Serialize corpus loads: finish (or let fail) the in-flight load before starting the next
2. If the previous load was abandoned, call finishExternalCorpusLoad (it resets staging on error) to clear the slot
3. Guard with your own in-progress flag around the start/append/finish sequence

Example fix

// before
ngram.startExternalCorpusLoad();
// other thread
ngram.startExternalCorpusLoad(); // throws
// after
std::lock_guard<std::mutex> g(corpus_load_mu);
ngram.startExternalCorpusLoad();

std::atomic<bool> load_in_progress{false};
if (load_in_progress.exchange(true)) { /* skip or queue */ return; }
ngram.startExternalCorpusLoad();

try { ngram.startExternalCorpusLoad(); } catch (const std::runtime_error& e) { if (strstr(e.what(), "in progress")) { /* wait for current load */ } else throw; }

When it happens

Trigger: Calling startExternalCorpusLoad() twice without an intervening finishExternalCorpusLoad() or a failed load that reset staging_sam_.

Common situations: Starting two async corpus downloads/loads concurrently against the same Ngram instance; a previous load that errored mid-way (e.g. empty corpus) leaving staging state; retry loops that call start again without finishing.

Related errors

• appendExternalCorpusTokens called without startExternalCorpu
• finishExternalCorpusLoad called without startExternalCorpusL
• Missing previous frame for delta payload
• External ngram corpus exceeds the remaining token budget ({m
• External corpus is empty — no tokens were loaded.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/dde5384d454cc553. Report an issue: GitHub.