{"record":{"id":"90799d432be18b32","repo":"huggingface/tokenizers","slug":"uninitialized-model","errorCode":null,"errorMessage":"Uninitialized Model","messagePattern":"Uninitialized Model","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"bindings/node/src/models.rs","lineNumber":126,"sourceCode":"        direction,\n      ),\n      None => pretokenized.tokenize(|normalized| guard.tokenize(normalized.get())),\n    }\n  }\n\n  fn token_to_id(&self, token: &str) -> Option<u32> {\n    self.model.as_ref()?.read().unwrap().token_to_id(token)\n  }\n\n  fn id_to_token(&self, id: u32) -> Option<String> {\n    self.model.as_ref()?.read().unwrap().id_to_token(id)\n  }\n\n  fn get_vocab(&self) -> HashMap<String, u32> {\n    self\n      .model\n      .as_ref()\n      .expect(\"Uninitialized Model\")\n      .read()\n      .unwrap()\n      .get_vocab()\n  }\n\n  fn get_vocab_size(&self) -> usize {\n    self\n      .model\n      .as_ref()\n      .expect(\"Uninitialized Model\")\n      .read()\n      .unwrap()\n      .get_vocab_size()\n  }\n\n  fn save(&self, folder: &Path, name: Option<&str>) -> tk::Result<Vec<PathBuf>> {\n    self\n      .model","sourceCodeStart":108,"sourceCodeEnd":144,"githubUrl":"https://github.com/huggingface/tokenizers/blob/6cfd9d385ca0ed91c10b49f0ce97d02cfde1b607/bindings/node/src/models.rs#L108-L144","documentation":"The Node binding's Model wrapper holds Option<model>; get_vocab (and sibling methods) call .expect(\"Uninitialized Model\") on it. If the wrapper was created with model: None — e.g. a model type instantiated empty instead of loaded from a vocab file — the expect panics and Node surfaces the panic as an error. The library throws because a vocabulary only exists once a concrete model (BPE/WordPiece/etc.) has been loaded.","triggerScenarios":"Calling get_vocab()/get_vocab_size() on a model wrapper whose inner model was never initialized, e.g. `new BPE()` with no constructor args or a model created through a code path that leaves the Option empty.","commonSituations":"Building a tokenizer in JS by constructing model classes directly without passing vocab/merges; deserializing a tokenizer.json with the binding and inspecting the raw model object; version mismatches between JS wrapper and native library where constructors changed.","solutions":["Initialize the model with its required data, e.g. BPE.from(vocab, merges) or load the whole tokenizer via tokenizer.fromJSON / from pretrained, before calling get_vocab.","Wrap the call in try/catch in JS; the panic surfaces as an exception you can handle and log.","Patch the binding (models.rs) to use self.model.as_ref().ok_or(\"Uninitialized Model\")? with a proper napi Error instead of expect()."],"exampleFix":"// before\nconst bpe = new BPE(); // empty\nbpe.getVocab(); // panics: Uninitialized Model\n\n// after\nconst bpe = await BPE.from(vocabObject, mergesArray);\nbpe.getVocab(); // { \"hello\": 0, ... }","handlingStrategy":"try-catch","validationCode":"// ensure model was created with vocab/merges before querying\nif (!bpe || !bpe.getVocab) throw new Error('model not initialized; use BPE.from(vocab, merges) or tokenizer.fromJSON');","typeGuard":null,"tryCatchPattern":"try { vocab = model.getVocab(); } catch (e) { if (String(e).includes('Uninitialized Model')) { model = await Tokenizer.fromJSON(config); vocab = model.model.getVocab(); } else throw e; }","preventionTips":["Always construct models with their required data (vocab, merges) via the provided from* methods","Prefer loading full tokenizers (from_pretrained / fromJSON) over assembling model objects manually","Keep native bindings and JS wrapper versions matched"],"tags":["rust","napi","node-bindings","uninitialized-state","model"],"backgroundTag":"internal-invariant-violation","analyzedSha":"6cfd9d385ca0ed91c10b49f0ce97d02cfde1b607","analyzedAt":"2026-09-09T11:43:25.027Z","contentChangedAt":"2026-09-09T11:43:25.027Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}