{"record":{"id":"7876fce430c70a96","repo":"krisk/Fuse","slug":"fuse-tokenize-regex-regex-lacks-the-global-fl","errorCode":null,"errorMessage":"[Fuse] tokenize regex ${regex} lacks the global flag; only the first match per text will be returned. Add the 'g' flag.","messagePattern":"\\[Fuse\\] tokenize regex (.+?) lacks the global flag; only the first match per text will be returned\\. Add the 'g' flag\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/search/token/analyzer.ts","lineNumber":25,"sourceCode":"}\n\ninterface AnalyzerOptions {\n  isCaseSensitive?: boolean\n  ignoreDiacritics?: boolean\n  tokenize?: RegExp | TokenizeFunction\n}\n\n// Includes \\p{M} (Mark) so combining marks stay attached to their base\n// letter — without it, scripts like Devanagari and NFD-normalized Latin\n// shatter (e.g. 'हिन्दी' → ['ह','न','द'], 'café'.normalize('NFD') → ['cafe']).\nconst DEFAULT_TOKEN = /[\\p{L}\\p{M}\\p{N}_]+/gu\n\nconst warned = new WeakSet<RegExp>()\n\nfunction warnNonGlobal(regex: RegExp): void {\n  if (process.env.NODE_ENV === 'development' && !warned.has(regex)) {\n    warned.add(regex)\n    console.warn(\n      `[Fuse] tokenize regex ${regex} lacks the global flag; only the ` +\n        `first match per text will be returned. Add the 'g' flag.`\n    )\n  }\n}\n\nfunction resolveTokenize(\n  tokenize: RegExp | TokenizeFunction | undefined\n): TokenizeFunction {\n  if (typeof tokenize === 'function') {\n    let validated = false\n    return (text: string): string[] => {\n      const result = tokenize(text)\n      if (process.env.NODE_ENV === 'development' && !validated) {\n        validated = true\n        if (\n          !Array.isArray(result) ||\n          result.some((t) => typeof t !== 'string')","sourceCodeStart":7,"sourceCodeEnd":43,"githubUrl":"https://github.com/krisk/Fuse/blob/edf2fb608eca0461508d1d71317e6e58309ffada/src/search/token/analyzer.ts#L7-L43","documentation":"Not a thrown error but a dev-mode-only console warning emitted by warnNonGlobal, called from resolveTokenize when a custom tokenize option is a RegExp without the global ('g') flag. String.prototype.match with a non-global regex returns only the first match, so texts would be tokenized into at most one token, silently degrading search recall. A WeakSet ensures the warning fires once per regex instance.","triggerScenarios":"Thrown at src/search/token/analyzer.ts:25 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Add the global flag to the regex: /pattern/g (add 'u' too if using unicode escapes, as the default /[\\p{L}\\p{M}\\p{N}_]+/gu does)","Reuse a single module-level regex with the g flag rather than constructing it inline","If the regex is intentionally non-global and single-match tokenization is desired, switch to a tokenize function to silence the warning"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"edf2fb608eca0461508d1d71317e6e58309ffada","analyzedAt":"2026-09-02T02:46:54.623Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}