antlr/antlr4 · error · IllegalArgumentException

Invalid tokens or characters at index "+pos+" in path '"+pat

Error message

Invalid tokens or characters at index "+pos+" in path '"+path+"'

What it means

While lexing an XPath path string, the XPathLexer hit a character sequence that matches no lexer rule (LexerNoViableAltException is rethrown instead of recovered). The resulting IllegalArgumentException reports the character position so you can fix the path syntax.

Source

Thrown at runtime/Java/src/org/antlr/v4/runtime/tree/xpath/XPath.java:106

			in = new ANTLRInputStream(new StringReader(path));
		}
		catch (IOException ioe) {
			throw new IllegalArgumentException("Could not read path: "+path, ioe);
		}
		XPathLexer lexer = new XPathLexer(in) {
			@Override
			public void recover(LexerNoViableAltException e) { throw e;	}
		};
		lexer.removeErrorListeners();
		lexer.addErrorListener(new XPathLexerErrorListener());
		CommonTokenStream tokenStream = new CommonTokenStream(lexer);
		try {
			tokenStream.fill();
		}
		catch (LexerNoViableAltException e) {
			int pos = lexer.getCharPositionInLine();
			String msg = "Invalid tokens or characters at index "+pos+" in path '"+path+"'";
			throw new IllegalArgumentException(msg, e);
		}

		List<Token> tokens = tokenStream.getTokens();
//		System.out.println("path="+path+"=>"+tokens);
		List<XPathElement> elements = new ArrayList<XPathElement>();
		int n = tokens.size();
		int i=0;
loop:
		while ( i<n ) {
			Token el = tokens.get(i);
			Token next = null;
			switch ( el.getType() ) {
				case XPathLexer.ROOT :
				case XPathLexer.ANYWHERE :
					boolean anywhere = el.getType() == XPathLexer.ANYWHERE;
					i++;
					next = tokens.get(i);
					boolean invert = next.getType()==XPathLexer.BANG;

View on GitHub (pinned to 7d5770395b)

Solutions

1. Restrict paths to supported syntax: '/', '//', '*', TOKEN_REF (uppercase), ruleRef (lowercase), and 'tokenName:"literal"'
2. Remove XML-XPath constructs like predicates, '..', or '@' from the path
3. Check the reported character index in the path string to locate the offending character

Example fix

// before
XPath.findAll(tree, "/statement/expr[1]", parser);

// after
XPath.findAll(tree, "/statement/expr", parser);

// allow only chars/syntax the tree XPath grammar supports
boolean ok = path.matches("[/A-Za-z0-9_:\"*]+") && !path.contains("..") && !path.contains("[");

try { XPath.findAll(tree, path, parser); } catch (IllegalArgumentException e) { /* e.getMessage() contains the bad index; log and reject the query */ }

When it happens

Trigger: Calling XPath.findAll(tree, xpath, parser) or new XPath(parser, path) with characters the XPath grammar does not support, e.g. '..', '@attr', '[1]', spaces, or unicode punctuation.

Common situations: Assuming ANTLR tree XPath supports XML XPath syntax (predicates, axes, parent navigation); copy-pasting XPath expressions from XML tooling into tree queries.

Understand the failure class

• Authentication and authorization failures — expired tokens, bad credentials, and missing scopes.

Related errors

• Missing path element at end of path
• Could not read path: "+path
• Unknowth path element "+el
• {word} at index {startIndex} isn't a valid token name
• {word} at index {startIndex} isn't a valid rule name

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/8dc8a12bb6ebfd2a. Report an issue: GitHub.