{"record":{"id":"04ca098a74f7614d","repo":"antlr/antlr4","slug":"invalid-tokens-or-characters-at-index-pos-in-pa","errorCode":null,"errorMessage":"Invalid tokens or characters at index ${pos} in path '${path}'","messagePattern":"Invalid tokens or characters at index (.+?) in path '(.+?)'","errorType":"exception","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"runtime/CSharp/src/Tree/Xpath/XPath.cs","lineNumber":114,"sourceCode":"                @in = new AntlrInputStream(new StringReader(path));\n            }\n            catch (IOException ioe)\n            {\n                throw new ArgumentException(\"Could not read path: \" + path, ioe);\n            }\n            XPathLexer lexer = new _XPathLexer_87(@in);\n            lexer.RemoveErrorListeners();\n            lexer.AddErrorListener(new XPathLexerErrorListener());\n            CommonTokenStream tokenStream = new CommonTokenStream(lexer);\n            try\n            {\n                tokenStream.Fill();\n            }\n            catch (LexerNoViableAltException e)\n            {\n                int pos = lexer.Column;\n                string msg = \"Invalid tokens or characters at index \" + pos + \" in path '\" + path + \"'\";\n                throw new ArgumentException(msg, e);\n            }\n            IList<IToken> tokens = tokenStream.GetTokens();\n            //\t\tSystem.out.println(\"path=\"+path+\"=>\"+tokens);\n            IList<XPathElement> elements = new List<XPathElement>();\n            int n = tokens.Count;\n            int i = 0;\n            while (i < n)\n            {\n                IToken el = tokens[i];\n                IToken next = null;\n                switch (el.Type)\n                {\n                    case XPathLexer.Root:\n                    case XPathLexer.Anywhere:\n                    {\n                        bool anywhere = el.Type == XPathLexer.Anywhere;\n                        i++;\n                        next = tokens[i];","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/CSharp/src/Tree/Xpath/XPath.cs#L96-L132","documentation":"XPath.Split lexes the XPath string with XPathLexer; when the lexer cannot match any rule at the current position it throws LexerNoViableAltException, which Split converts into ArgumentException naming the column of the offending character. In other words, the XPath query contains characters or tokens that the XPath grammar does not support.","triggerScenarios":"new XPath(parser, \"//declaration/*bad\") — invalid characters, unsupported operators, stray quotes, or whitespace the XPath lexer rejects. Valid XPath here is the ANTLR parse-tree subset: '/', '//', '*', word, '!', and rule-or-token names.","commonSituations":"Assuming W3C XPath syntax (predicates like [1], '@attr', '..') works — ANTLR's XPath is a tiny subset; copy-pasting real XPath expressions from elsewhere; grammar-specific names not matching the parser's rule/token names.","solutions":["Rewrite the query using the supported subset: '/', '//', '*', identifier, and '!' anywhere (e.g. \"//funcDecl\", \"/translationUnit/decl/*\").","Remove unsupported constructs (predicates, attributes, parent axes) and do the fine-grained filtering in C# over the returned nodes.","Verify every name in the path against parser.RuleNames / Vocabulary so the elements resolve after lexing.","Catch ArgumentException around XPath construction to report the bad query with its index when queries come from users."],"exampleFix":"// before\nvar xp = new XPath(parser, \"//funcDecl[@name='foo']\"); // predicates unsupported\n\n// after\nvar xp = new XPath(parser, \"//funcDecl\");\nforeach (var ctx in xp.Evaluate(tree).OfType<FuncDeclContext>())\n    if (ctx.name?.GetText() == \"foo\") { /* ... */ }","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { var xp = new XPath(parser, path); } catch (ArgumentException ex) when (ex.Message.Contains(\"Invalid tokens or characters\")) { /* surface 'unsupported XPath syntax', suggest the ANTLR subset: / // * word ! */ }","preventionTips":["Learn the supported XPath subset: '/', '//', '*', identifiers, '!'; no predicates, attributes, or W3C axes.","Do fine-grained node filtering in C# over Evaluate() results instead of encoding it in the path.","Validate user-supplied queries against a whitelist of allowed characters before constructing XPath."],"tags":["antlr","csharp","xpath","lexer","syntax"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}