{"record":{"id":"c9d79bb880ab7211","repo":"sgl-project/sglang","slug":"external-corpus-corpus-id-already-exists-rem","errorCode":null,"errorMessage":"External corpus '${corpus_id}' already exists. Remove it before adding a new corpus with the same id.","messagePattern":"External corpus '(.+?)' already exists\\. Remove it before adding a new corpus with the same id\\.","errorType":"exception","errorClass":"std::runtime_error","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/jit/csrc/ngram_corpus/ngram.cpp","lineNumber":98,"sourceCode":"  if (!staging_sam_) {\n    throw std::runtime_error(\"appendExternalCorpusTokens called without startExternalCorpusLoad\");\n  }\n  staging_sam_->appendTokens(tokens);\n}\n\nvoid Ngram::finishExternalCorpusLoad(const std::string& corpus_id) {\n  if (!staging_sam_) {\n    throw std::runtime_error(\"finishExternalCorpusLoad called without startExternalCorpusLoad\");\n  }\n  staging_sam_->finalize();\n  if (staging_sam_->empty()) {\n    staging_sam_.reset();\n    throw std::runtime_error(\"External corpus is empty — no tokens were loaded.\");\n  }\n  // Only lock briefly to install the completed SAM.\n  std::unique_lock<std::mutex> lock(mutex_);\n  if (sams_.find(corpus_id) != sams_.end()) {\n    throw std::runtime_error(\n        \"External corpus '\" + corpus_id + \"' already exists. Remove it before adding a new corpus with the same id.\");\n  }\n  sams_.emplace(corpus_id, std::move(staging_sam_));\n}\n\nvoid Ngram::removeExternalCorpus(const std::string& corpus_id) {\n  std::unique_lock<std::mutex> lock(mutex_);\n  sams_.erase(corpus_id);\n}\n\nvoid Ngram::resetStagingSam() {\n  // staging_sam_ is only accessed from the loading thread — no lock needed.\n  staging_sam_.reset();\n}\n\nvoid Ngram::clearExternalCorpus() {\n  std::unique_lock<std::mutex> lock(mutex_);\n  sams_.clear();","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/jit/csrc/ngram_corpus/ngram.cpp#L80-L116","documentation":"Corpora are stored per corpus_id in a map; installing a new corpus under an id that already exists is rejected to avoid silently replacing data. The staged automaton stays installed... actually staging was already moved; the throw happens under mutex before emplace.","triggerScenarios":"finishExternalCorpusLoad(corpus_id) where corpus_id already has an entry in sams_ (a previous successful load with the same id).","commonSituations":"Re-running a corpus load script without removeExternalCorpus first; reloading/refreshing a corpus on a long-lived server; id collisions between datasets.","solutions":["Call removeExternalCorpus(corpus_id) before loading the same id again","Use a unique corpus id per load (e.g. versioned: 'wiki-v2')","Check hasExternalCorpus(corpus_id)/equivalent API before finishing"],"exampleFix":"// before\nngram.startExternalCorpusLoad();\n/* append */ ngram.finishExternalCorpusLoad(\"wiki\"); // 'wiki' exists\n// after\nif (has_corpus(\"wiki\")) ngram.removeExternalCorpus(\"wiki\");\nngram.startExternalCorpusLoad();\n/* append */ ngram.finishExternalCorpusLoad(\"wiki\");","handlingStrategy":"try-catch","validationCode":"// if an exists-check API is available:\nif (ngram.hasExternalCorpus(id)) ngram.removeExternalCorpus(id);\nngram.finishExternalCorpusLoad(id);","typeGuard":null,"tryCatchPattern":"try { ngram.finishExternalCorpusLoad(id); } catch (const std::runtime_error& e) {\n  if (std::string(e.what()).find(\"already exists\") != std::string::npos) { ngram.removeExternalCorpus(id); /* reload */ }\n}","preventionTips":["Use versioned corpus ids (wiki-v2) on refresh","Always removeExternalCorpus before reloading the same id"],"tags":["ngram","corpus-loading","duplicate-key"],"backgroundTag":"duplicate-resource-id","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}