huggingface/tokenizers · error · Error

WASI binding not found and NAPI_RS_FORCE_WASI is set to…

Error message

WASI binding not found and NAPI_RS_FORCE_WASI is set to error

What it means

When NAPI_RS_FORCE_WASI is set, the loader refuses to use the plain native binding and insists on loading a WASI binding (./tokenizers.wasi.cjs or the tokenizers-wasm32-wasi package). If neither WASI binding can be required and the variable is set to the exact value 'error', the loader throws this error (wrapping the underlying require failure as `cause`). It exists so users can force WASI mode and hard-fail instead of silently falling back.

Solutions

  1. Install the WASI binding: `npm i tokenizers-wasm32-wasi` (ensure optional dependencies aren't skipped)
  2. Remove NAPI_RS_FORCE_WASI (or set it to anything other than 'error', e.g. 'warn') so the loader can use the native binding
  3. Check error.cause to see the underlying require failure (missing module, syntax error) and fix that

Example fix

// before
NAPI_RS_FORCE_WASI=error node app.js
// after
npm i tokenizers-wasm32-wasi
# or
unset NAPI_RS_FORCE_WASI
Defensive patterns

Strategy: validation

Validate before calling

if (process.env.NAPI_RS_FORCE_WASI === 'error') {
  let ok = false;
  try { require('tokenizers-wasm32-wasi'); ok = true; } catch {}
  if (!ok) console.error('NAPI_RS_FORCE_WASI=error but WASI binding is not installed');
}

Type guard

function wasiBindingAvailable() {
  try { require.resolve('tokenizers-wasm32-wasi'); return true; } catch { return false; }
}

Try / catch

try {
  const tokenizers = require('tokenizers');
} catch (err) {
  if (String(err.message).includes('NAPI_RS_FORCE_WASI')) {
    console.error('WASI binding missing; install tokenizers-wasm32-wasi or unset NAPI_RS_FORCE_WASI');
  }
  throw err;
}

Prevention

When it happens

Trigger: process.env.NAPI_RS_FORCE_WASI === 'error' at require time and both require('./tokenizers.wasi.cjs') and require('tokenizers-wasm32-wasi') fail (files/packages absent or unloadable).

Common situations: Setting NAPI_RS_FORCE_WASI=error in an environment (e.g. serverless/WebAssembly targets) where the wasm32-wasi optional package was never installed; Docker images that strip optional dependencies; typos leaving the variable set to 'error' unintentionally.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

AI-assisted analysis of huggingface/tokenizers@6cfd9d385c (2026-09-09). Data as JSON: /api/errors/a3e90f6f6e04d8a9. Report an issue: GitHub.

Appendix: source

Thrown at bindings/node/index.js:722

        if (!wasiBindingError) {
          wasiBindingError = err
        } else {
          wasiBindingError.cause = err
        }
        loadErrors.push(err)
      }
    }
  }
  if (process.env.NAPI_RS_FORCE_WASI === 'error' && !wasiBinding) {
    const error = new Error('WASI binding not found and NAPI_RS_FORCE_WASI is set to error')
    error.cause = wasiBindingError
    throw error
  }
}

if (!nativeBinding) {
  if (loadErrors.length > 0) {
    throw new Error(
      `Cannot find native binding. ` +
        `npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). ` +
        'Please try `npm i` again after removing both package-lock.json and node_modules directory.',
      {
        cause: loadErrors.reduce((err, cur) => {
          cur.cause = err
          return cur
        }),
      },
    )
  }
  throw new Error(`Failed to load native binding`)
}

module.exports = nativeBinding
module.exports.AddedToken = nativeBinding.AddedToken
module.exports.BPE = nativeBinding.BPE
module.exports.Bpe = nativeBinding.Bpe

View on GitHub (pinned to 6cfd9d385c)