{"record":{"id":"754d2282b8c33ff4","repo":"huggingface/tokenizers","slug":"cannot-find-native-binding-npm-has-a-bug-related","errorCode":null,"errorMessage":"Cannot find native binding. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.","messagePattern":"Cannot find native binding\\. npm has a bug related to optional dependencies \\(https://github\\.com/npm/cli/issues/4828\\)\\. Please try `npm i` again after removing both package-lock\\.json and node_modules directory\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"critical","filePath":"bindings/node/index.js","lineNumber":727,"sourceCode":"        loadErrors.push(err)\n      }\n    }\n  }\n  if (process.env.NAPI_RS_FORCE_WASI === 'error' && !wasiBinding) {\n    const error = new Error('WASI binding not found and NAPI_RS_FORCE_WASI is set to error')\n    error.cause = wasiBindingError\n    throw error\n  }\n}\n\nif (!nativeBinding) {\n  if (loadErrors.length > 0) {\n    throw new Error(\n      `Cannot find native binding. ` +\n        `npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). ` +\n        'Please try `npm i` again after removing both package-lock.json and node_modules directory.',\n      {\n        cause: loadErrors.reduce((err, cur) => {\n          cur.cause = err\n          return cur\n        }),\n      },\n    )\n  }\n  throw new Error(`Failed to load native binding`)\n}\n\nmodule.exports = nativeBinding\nmodule.exports.AddedToken = nativeBinding.AddedToken\nmodule.exports.BPE = nativeBinding.BPE\nmodule.exports.Bpe = nativeBinding.Bpe\nmodule.exports.Decoder = nativeBinding.Decoder\nmodule.exports.Encoding = nativeBinding.Encoding\nmodule.exports.JsEncoding = nativeBinding.JsEncoding\nmodule.exports.Model = nativeBinding.Model\nmodule.exports.Normalizer = nativeBinding.Normalizer","sourceCodeStart":709,"sourceCodeEnd":745,"githubUrl":"https://github.com/huggingface/tokenizers/blob/6cfd9d385ca0ed91c10b49f0ce97d02cfde1b607/bindings/node/index.js#L709-L745","documentation":"This is the loader's final fallback failure: no platform native binding could be loaded at all (all require attempts failed and were collected in loadErrors). The message points at a known npm bug with optional dependencies (npm/cli#4828) where node_modules gets an incomplete set of platform-specific packages, so it advises a clean reinstall. All underlying per-platform require errors are chained via error.cause.","triggerScenarios":"process.arch/process.platform matches no built binding, local .node files are missing, and platform packages like tokenizers-darwin-arm64/tokenizers-linux-x64-gnu etc. cannot be required, leaving nativeBinding null with loadErrors non-empty when bindings/node/index.js is required.","commonSituations":"npm bug #4828 dropping optional platform packages after `npm ci` from a lockfile created on another OS; running on an unusual platform (musl/Alpine, OpenHarmony) without the matching optional package; `npm i --no-optional`; switching Node/OS without reinstalling.","solutions":["Delete node_modules and package-lock.json, then run `npm i` again (per the npm/cli#4828 workaround)","Inspect error.cause chain to see which specific binding require failed and why","Install the platform-specific optional package explicitly, e.g. `npm i tokenizers-linux-x64-gnu` (or your platform's package)","If on an unsupported OS/arch, check `process.platform`/`process.arch`; the loader will never find a binding there (see 'Unsupported OS/architecture' load errors in the cause chain)"],"exampleFix":"// before: binding missing after partial install\nconst t = require('tokenizers') // throws Cannot find native binding...\n// after\n// rm -rf node_modules package-lock.json && npm i\nconst t = require('tokenizers') // works","handlingStrategy":"fallback","validationCode":"function canLoadBinding() {\n  try { require('tokenizers'); return true; } catch (err) {\n    console.error('Binding load failed:', err.cause ?? err);\n    return false;\n  }\n}","typeGuard":"function isTokenizersLoaded(mod) {\n  return !!mod && typeof mod === 'object' && typeof mod.encode !== 'undefined' || !!(mod && mod.AddedToken);\n}","tryCatchPattern":"let tokenizers;\ntry {\n  tokenizers = require('tokenizers');\n} catch (err) {\n  const firstCause = err.cause;\n  console.error('Native binding missing. Underlying errors:', firstCause);\n  console.error('Fix: rm -rf node_modules package-lock.json && npm i');\n  throw err;\n}","preventionTips":["Don't install with --no-optional or --omit=optional; platform bindings are optional deps","After changing Node versions or OS (e.g. rebuilding Docker images), always do a clean reinstall","Walk the error.cause chain to identify exactly which platform package failed before guessing","If using npm < 9, consider upgrading to avoid npm/cli#4828"],"tags":["nodejs","npm","optional-dependencies","native-binding"],"backgroundTag":"missing-optional-dependency","analyzedSha":"6cfd9d385ca0ed91c10b49f0ce97d02cfde1b607","analyzedAt":"2026-09-09T11:43:25.027Z","contentChangedAt":"2026-09-09T11:43:25.027Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}