{"record":{"id":"a6113eb15bf48abd","repo":"markedjs/marked","slug":"infinite-loop-on-byte-byte","errorCode":null,"errorMessage":"Infinite loop on byte: ${byte}","messagePattern":"Infinite loop on byte: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"src/Lexer.ts","lineNumber":487,"sourceCode":"        }\n        continue;\n      }\n\n      if (src) {\n        this.infiniteLoopError(src.charCodeAt(0));\n        break;\n      }\n    }\n\n    return tokens;\n  }\n\n  private infiniteLoopError(byte: number) {\n    const errMsg = 'Infinite loop on byte: ' + byte;\n    if (this.options.silent) {\n      console.error(errMsg);\n    } else {\n      throw new Error(errMsg);\n    }\n  }\n}\n","sourceCodeStart":469,"sourceCodeEnd":491,"githubUrl":"https://github.com/markedjs/marked/blob/9552b6bbca7f587f96b7249c5833358a17a18e2b/src/Lexer.ts#L469-L491","documentation":"Thrown by the lexer's loop guard in blockTokens and inlineTokens. The lexer tracks the smallest src.length it has seen (srcLength); on each iteration, if src did not get strictly shorter than that minimum, marked assumes no tokenizer consumed any input and calls infiniteLoopError(src.charCodeAt(0)) (Lexer.ts:114-120, 337-343, 285-288, 473-476). The byte in the message is the char code of the first un-consumed character. This guard prevents a true infinite loop that would hang the process. With built-in tokenizers alone it is essentially unreachable because the text/inlineText tokenizers match at least one character; it almost always indicates a custom tokenizer extension that returned a token whose raw is empty (so src.substring(token.raw.length) leaves src unchanged), a zero-width regex match, or a start function that mis-clips src.","triggerScenarios":"A custom tokenizer extension whose returned token has raw:'' (or raw shorter than claimed) so src never shrinks (Lexer.ts:126, 355); an extension regex that matches zero-width (using * where + was needed); a start function that mis-clips src for the paragraph tokenizer yielding an empty match; a tokenizer that returns a token on some calls and undefined on others while built-in fallbacks also fail on that byte.","commonSituations":"Writing a tokenizer extension and forgetting to set raw (or setting it to ''); computing raw from the wrong capture group; an extension regex using * instead of + producing a zero-width match; porting an extension from an older marked where the contract differed.","solutions":["Audit every custom tokenizer extension: ensure the returned token's raw is the exact substring consumed and has length > 0.","Make the tokenizer return undefined when it should NOT match, so built-in tokenizers get a chance to consume the byte.","Reproduce with the offending input in isolation and log src.length per iteration to find which extension stalls.","If using a start function, verify it returns a sane index (>= 0) and does not cause clipping that yields an empty match.","Temporarily set silent:true to log instead of throw while debugging, but do not ship that."],"exampleFix":"// before — raw is empty, src never shrinks => infinite loop\nmarked.use({\n  extensions: [{\n    name: 'tag',\n    level: 'inline',\n    tokenizer(src) {\n      const m = /^\\[\\[(.*?)\\]\\]/.exec(src);\n      if (!m) return;\n      return { type: 'tag', raw: '', text: m[1], tokens: [] }; // BUG: raw should be m[0]\n    },\n    renderer(token) { return `<span>${token.text}</span>`; }\n  }]\n});\n\n// after — raw is the full match, src advances\n    return { type: 'tag', raw: m[0], text: m[1], tokens: [] };","handlingStrategy":"try-catch","validationCode":"function assertExtensionConsumes(ext, samples) {\n  for (const src of samples) {\n    const tok = ext.tokenizer.call({ lexer: null }, src, []);\n    if (tok && (!tok.raw || tok.raw.length === 0)) {\n      throw new Error('extension \"' + ext.name + '\" matched but consumed 0 chars on input ' + JSON.stringify(src));\n    }\n  }\n}\nassertExtensionConsumes(myInlineExt, ['[[x]]', 'no match here', 'plain text']);","typeGuard":"function tokenConsumesInput(tok, srcBefore, srcAfter) {\n  return !!tok && typeof tok.raw === 'string' && tok.raw.length > 0 && srcAfter.length < srcBefore.length;\n}","tryCatchPattern":"try {\n  return marked.parse(userInput);\n} catch (e) {\n  if (e instanceof Error && e.message.startsWith('Infinite loop on byte:')) {\n    logger.error({ byte: e.message, input: userInput }, 'marked infinite loop guard tripped');\n    throw new Error('Unable to render markdown: tokenizer stalled');\n  }\n  throw e;\n}","preventionTips":["Always set token.raw to the exact matched substring and ensure its length > 0 before returning.","Return undefined (not an empty token) when your extension does not match.","Write a tokenizer-extension unit test that feeds tricky inputs and asserts src strictly shrinks on each successful call.","Prefer + over * in extension regexes to avoid zero-width matches.","Never retry the same input unchanged after this error - fix the extension or transform the input."],"tags":["lexer","runtime","extensions","tokenizer","infinite-loop","debugging"],"backgroundTag":null,"analyzedSha":"9552b6bbca7f587f96b7249c5833358a17a18e2b","analyzedAt":"2026-08-13T04:27:55.513Z","schemaVersion":2},"datasetVersion":"2026-08-13T09:17:06.757Z"}