huggingface/tokenizers · error · Error

Native binding package version mismatch, expected…

Error message

Native binding package version mismatch, expected 0.15.3-dev0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.

What it means

This error is thrown by requireNative in bindings/node/index.js when the optional platform-specific native binding package (here tokenizers-android-arm64) has a version that does not match the version the main tokenizers package was compiled against (0.15.3-dev0). The main package pins its native binding packages so the JS wrapper and the prebuilt .node binary always agree; a mismatch means the loaded binary may be incompatible. The check is enforced only when the environment variable NAPI_RS_ENFORCE_VERSION_CHECK is set to a non-empty value other than '0'.

Solutions

Run the package manager's reinstall (e.g. `npm install`, `yarn install`, or `pnpm install`) so the matching tokenizers-android-arm64@0.15.3-dev0 optional dependency is installed.
Explicitly install the correct binding version: `npm i tokenizers-android-arm64@0.15.3-dev0`.
Clear lockfile/caches if a stale version keeps resolving (`npm cache clean --force`, delete lockfile, reinstall).
Unset NAPI_RS_ENFORCE_VERSION_CHECK (or set it to '0') to bypass the check only if you accept the version mismatch.

Example fix

// before
npm ls tokenizers-android-arm64  # shows 0.15.2
// after
rm -rf node_modules package-lock.json && npm install
# tokenizers-android-arm64 now resolves to 0.15.3-dev0

Defensive patterns

Strategy: try-catch

Validate before calling

const ver = (() => { try { return require('tokenizers-android-arm64/package.json').version } catch { return null } })();
if (ver && ver !== '0.15.3-dev0' && process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0') {
  throw new Error(`tokenizers-android-arm64 version ${ver} != 0.15.3-dev0; reinstall deps before importing tokenizers`);
}

Type guard

function isBindingVersionOk(pkgName, expected = '0.15.3-dev0') {
  try {
    const v = require(`${pkgName}/package.json`).version;
    return typeof v === 'string' && v === expected;
  } catch {
    return false;
  }
}

Try / catch

try {
  const { Tokenizer } = require('tokenizers');
} catch (e) {
  if (String(e.message).includes('version mismatch')) {
    console.error('Native binding version mismatch: run `npm install` to reinstall platform bindings');
  } else {
    throw e;
  }
}

Prevention

After bumping tokenizers, always delete node_modules and the lockfile (or run npm ci) so optional platform bindings are refreshed.
Do not pin/override tokenizers-* platform packages to different versions in package.json.
Check installed binding versions in CI before running tests that import tokenizers.
Avoid `npm install --omit=optional` for apps that need the native binding.
Know that NAPI_RS_ENFORCE_VERSION_CHECK controls strictness; leave it set in CI to catch drift early.

When it happens

Trigger: Requiring the tokenizers module on an android-arm64 platform while the installed tokenizers-android-arm64 package's version !== '0.15.3-dev0', with NAPI_RS_ENFORCE_VERSION_CHECK set (e.g. to '1'). The thrown error is caught inside requireNative and pushed to loadErrors, typically surfacing later when no binding could be loaded.

Common situations: Partial installs after upgrading/downgrading tokenizers without reinstalling optional dependencies; lockfile or registry caching an old binding package; manually installed or hoisted mismatched binding versions; pnpm/yarn resolution pinning a stale optional dependency.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of huggingface/tokenizers@6cfd9d385c (2026-09-09). Data as JSON: /api/errors/d82ceb5329dbf5fe. Report an issue: GitHub.

Appendix: source

Thrown at bindings/node/index.js:85

    } catch (err) {
      loadErrors.push(err)
    }
  } else if (process.platform === 'android') {
    if (process.arch === 'arm64') {
      try {
        return require('./tokenizers.android-arm64.node')
      } catch (e) {
        loadErrors.push(e)
      }
      try {
        const binding = require('tokenizers-android-arm64')
        const bindingPackageVersion = require('tokenizers-android-arm64/package.json').version
        if (
          bindingPackageVersion !== '0.15.3-dev0' &&
          process.env.NAPI_RS_ENFORCE_VERSION_CHECK &&
          process.env.NAPI_RS_ENFORCE_VERSION_CHECK !== '0'
        ) {
          throw new Error(
            `Native binding package version mismatch, expected 0.15.3-dev0 but got ${bindingPackageVersion}. You can reinstall dependencies to fix this issue.`,
          )
        }
        return binding
      } catch (e) {
        loadErrors.push(e)
      }
    } else if (process.arch === 'arm') {
      try {
        return require('./tokenizers.android-arm-eabi.node')
      } catch (e) {
        loadErrors.push(e)
      }
      try {
        const binding = require('tokenizers-android-arm-eabi')
        const bindingPackageVersion = require('tokenizers-android-arm-eabi/package.json').version
        if (
          bindingPackageVersion !== '0.15.3-dev0' &&

View on GitHub (pinned to 6cfd9d385c)