{"id":"abdf6acc723f16f1","repo":"eslint/eslint","slug":"ast-is-missing-the-tokens-array","errorCode":null,"errorMessage":"AST is missing the tokens array.","messagePattern":"AST is missing the tokens array\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"lib/languages/js/source-code/source-code.js","lineNumber":54,"sourceCode":"// Private\n//------------------------------------------------------------------------------\n\nconst commentParser = new ConfigCommentParser();\n\n/**\n * Validates that the given AST has the required information.\n * @param {ASTNode} ast The Program node of the AST to check.\n * @throws {TypeError} If the AST doesn't contain the correct information.\n * @returns {void}\n * @private\n */\nfunction validate(ast) {\n\tif (!ast) {\n\t\tthrow new TypeError(`Unexpected empty AST. (${ast})`);\n\t}\n\n\tif (!ast.tokens) {\n\t\tthrow new TypeError(\"AST is missing the tokens array.\");\n\t}\n\n\tif (!ast.comments) {\n\t\tthrow new TypeError(\"AST is missing the comments array.\");\n\t}\n\n\tif (!ast.loc) {\n\t\tthrow new TypeError(\"AST is missing location information.\");\n\t}\n\n\tif (!ast.range) {\n\t\tthrow new TypeError(\"AST is missing range information\");\n\t}\n}\n\n/**\n * Retrieves globals for the given ecmaVersion.\n * @param {number} ecmaVersion The version to retrieve globals for.","sourceCodeStart":36,"sourceCodeEnd":72,"githubUrl":"https://github.com/eslint/eslint/blob/f131c034ad91bbf06bcbb6b5e931447a8e419a46/lib/languages/js/source-code/source-code.js#L36-L72","documentation":"Thrown as a TypeError by validate(ast) (lib/languages/js/source-code/source-code.js:54) when the AST has no `tokens` array. SourceCode indexes tokens for getTokens/getTokenAfter/range lookups; the parser must emit a tokens array (the parserOptions `tokens: true` is set by the JS language at index.js:252).","triggerScenarios":"A custom parser that does not populate `ast.tokens`, or a parser wrapper that strips tokens to save memory. The standard espree parser always emits tokens when `tokens: true`.","commonSituations":"Plugging in a parser whose options don't include tokens:true, or that returns an AST shape from a different parser library (e.g. @babel/parser's AST without token collection, or a TypeScript AST which has no tokens concept).","solutions":["Enable tokens in the parser: pass `tokens: true` (the JS language already does this at index.js:252; ensure your wrapper forwards it).","Map the parser's token stream into `ast.tokens = [...]` before returning.","Use a parser compatible with ESLint's expected AST (espree, or a wrapper that produces the ESTree+tokens shape)."],"exampleFix":"// before\nreturn { ast: babelParse(text, { ranges: true }) }; // no .tokens\n\n// after\nconst ast = babelParse(text, { tokens: true, ranges: true });\nreturn { ast };","handlingStrategy":"validation","validationCode":"if (!Array.isArray(ast.tokens)) {\n  throw new Error('Parser AST is missing the tokens array; enable tokens:true');\n}","typeGuard":"const hasTokens = ast => Array.isArray(ast.tokens);","tryCatchPattern":null,"preventionTips":["Forward tokens:true to the underlying parser.","Map the parser's token list to ast.tokens before returning.","Use a parser whose output matches the ESLint AST contract."],"tags":["parser","ast","tokens","custom-parser"],"analyzedSha":"f131c034ad91bbf06bcbb6b5e931447a8e419a46","analyzedAt":"2026-08-03T19:53:24.025Z","schemaVersion":2}