krisk/Fuse · error

[Fuse] tokenize regex ${regex} lacks the global flag; only t

Error message

[Fuse] tokenize regex ${regex} lacks the global flag; only the first match per text will be returned. Add the 'g' flag.

What it means

Not a thrown error but a dev-mode-only console warning emitted by warnNonGlobal, called from resolveTokenize when a custom tokenize option is a RegExp without the global ('g') flag. String.prototype.match with a non-global regex returns only the first match, so texts would be tokenized into at most one token, silently degrading search recall. A WeakSet ensures the warning fires once per regex instance.

Source

Thrown at src/search/token/analyzer.ts:25

}

interface AnalyzerOptions {
  isCaseSensitive?: boolean
  ignoreDiacritics?: boolean
  tokenize?: RegExp | TokenizeFunction
}

// Includes \p{M} (Mark) so combining marks stay attached to their base
// letter — without it, scripts like Devanagari and NFD-normalized Latin
// shatter (e.g. 'हिन्दी' → ['ह','न','द'], 'café'.normalize('NFD') → ['cafe']).
const DEFAULT_TOKEN = /[\p{L}\p{M}\p{N}_]+/gu

const warned = new WeakSet<RegExp>()

function warnNonGlobal(regex: RegExp): void {
  if (process.env.NODE_ENV === 'development' && !warned.has(regex)) {
    warned.add(regex)
    console.warn(
      `[Fuse] tokenize regex ${regex} lacks the global flag; only the ` +
        `first match per text will be returned. Add the 'g' flag.`
    )
  }
}

function resolveTokenize(
  tokenize: RegExp | TokenizeFunction | undefined
): TokenizeFunction {
  if (typeof tokenize === 'function') {
    let validated = false
    return (text: string): string[] => {
      const result = tokenize(text)
      if (process.env.NODE_ENV === 'development' && !validated) {
        validated = true
        if (
          !Array.isArray(result) ||
          result.some((t) => typeof t !== 'string')

View on GitHub (pinned to edf2fb608e)

Solutions

1. Add the global flag to the regex: /pattern/g (add 'u' too if using unicode escapes, as the default /[\p{L}\p{M}\p{N}_]+/gu does)
2. Reuse a single module-level regex with the g flag rather than constructing it inline
3. If the regex is intentionally non-global and single-match tokenization is desired, switch to a tokenize function to silence the warning

When it happens

Trigger: Thrown at src/search/token/analyzer.ts:25 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of krisk/Fuse@edf2fb608e (2026-09-02). Data as JSON: /api/errors/7876fce430c70a96. Report an issue: GitHub.