eslint/eslint · critical

AST is missing the tokens array.

Error message

AST is missing the tokens array.

What it means

Thrown as a TypeError by validate(ast) (lib/languages/js/source-code/source-code.js:54) when the AST has no `tokens` array. SourceCode indexes tokens for getTokens/getTokenAfter/range lookups; the parser must emit a tokens array (the parserOptions `tokens: true` is set by the JS language at index.js:252).

Source

Thrown at lib/languages/js/source-code/source-code.js:54

// Private
//------------------------------------------------------------------------------

const commentParser = new ConfigCommentParser();

/**
 * Validates that the given AST has the required information.
 * @param {ASTNode} ast The Program node of the AST to check.
 * @throws {TypeError} If the AST doesn't contain the correct information.
 * @returns {void}
 * @private
 */
function validate(ast) {
	if (!ast) {
		throw new TypeError(`Unexpected empty AST. (${ast})`);
	}

	if (!ast.tokens) {
		throw new TypeError("AST is missing the tokens array.");
	}

	if (!ast.comments) {
		throw new TypeError("AST is missing the comments array.");
	}

	if (!ast.loc) {
		throw new TypeError("AST is missing location information.");
	}

	if (!ast.range) {
		throw new TypeError("AST is missing range information");
	}
}

/**
 * Retrieves globals for the given ecmaVersion.
 * @param {number} ecmaVersion The version to retrieve globals for.

View on GitHub (pinned to f131c034ad)

Solutions

1. Enable tokens in the parser: pass `tokens: true` (the JS language already does this at index.js:252; ensure your wrapper forwards it).
2. Map the parser's token stream into `ast.tokens = [...]` before returning.
3. Use a parser compatible with ESLint's expected AST (espree, or a wrapper that produces the ESTree+tokens shape).

Example fix

// before
return { ast: babelParse(text, { ranges: true }) }; // no .tokens

// after
const ast = babelParse(text, { tokens: true, ranges: true });
return { ast };

if (!Array.isArray(ast.tokens)) {
  throw new Error('Parser AST is missing the tokens array; enable tokens:true');
}

const hasTokens = ast => Array.isArray(ast.tokens);

When it happens

Trigger: A custom parser that does not populate `ast.tokens`, or a parser wrapper that strips tokens to save memory. The standard espree parser always emits tokens when `tokens: true`.

Common situations: Plugging in a parser whose options don't include tokens:true, or that returns an AST shape from a different parser library (e.g. @babel/parser's AST without token collection, or a TypeScript AST which has no tokens concept).

Related errors

• Unexpected empty AST. (${ast})
• AST is missing the comments array.
• AST is missing location information.
• AST is missing range information
• Key "languageOptions": ${error.message}

AI-assisted analysis of eslint/eslint@f131c034ad (2026-08-03). Data as JSON: /data/errors/abdf6acc723f16f1.json. Report an issue: GitHub.