ErrLookup › huggingface/tokenizers
huggingface/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production · Rust · 141 source files
Analyzed at 6cfd9d385c on 2026-09-09. 26 documented errors.
| Code / Message | Type | Severity | Tags |
|---|---|---|---|
| Native binding package version mismatch, expected… | exception | error | version-mismatch, native-binding, installation, nodejs |
| Helper | panic | error | serde, deserialization, tokenizer, version-mismatch, rust |
| You don't seem to have the required protobuf file, in order… | exception | error | python, protobuf, missing-dependency, sentencepiece |
| WASI binding not found and NAPI_RS_FORCE_WASI is set to… | exception | error | wasi, env-var, napi-rs, nodejs |
| NormalizedString bad split | panic | error | rust, unicode, pre-tokenization, internal-invariant, tokenizer |
| Deprecated in | console | warning | python, deprecation, tokenizers, pyo3 |
| Uninitialized Model | panic | error | rust, napi, node-bindings, uninitialized-state, model |
| Cannot build Piece from string | validation | error | rust, tokenizers, template-processing, parsing |
| Cannot find native binding. npm has a bug related to… | exception | critical | nodejs, npm, optional-dependencies, native-binding |
| AddedVocabulary bad split | panic | error | rust, added-tokens, normalization, internal-invariant, tokenizer |
| encode_batch: `inputs` can't be `None` | validation | error | python, null-argument, tokenizers |
| is not a known unicode normalizer. Available are | validation | error | python, normalizer, invalid-enum-value, configuration |
| Failed to load native binding | exception | error | nodejs, native-binding, module-init, napi-rs |
| likelihood is NAN. Input sentence may be too long. | panic | error | rust, training, unigram, numerical-overflow, panic |
| You're trying to run a `Unigram` model but you're file was… | exception | error | python, sentencepiece, model-mismatch, unigram |
| sep_token not found in the vocabulary | exception | error | python, tokenizer, vocabulary, config-mismatch |
| encode: `sequence` can't be `None` | validation | error | python, null-argument, tokenizers, validation |
| Error in offsets | error_code | error | node, offsets, range-error, argument-out-of-range |
| cls_token not found in the vocabulary | exception | error | python, tokenizer, vocabulary, config-mismatch |
| Uninitialized Encoding | panic | error | node, rust, napi, uninitialized, panic |
| We couldn't import IPython utils for html display. Are you… | exception | error | python, import-error, ipython, optional-dependency, visualization |
| async_encode_batch: `inputs` can't be `None` | validation | error | python, async, null-argument, tokenizers |
| None input is not valid. Should be a list of integers. | validation | error | python, null-argument, tokenizers |
| async_encode_batch_fast: `inputs` can't be `None` | validation | error | python, async, null-argument, tokenizers |
| None input is not valid. Should be list of list of integers. | validation | error | python, null-argument, tokenizers |
| async_decode_batch: `sequences` can't be `None` | validation | error | python, async, null-argument, tokenizers |