antlr/antlr4 · error · Exception

%s at index %d isn't a valid token name

Error message

%s at index %d isn't a valid token name

What it means

Raised by XPath.getXPathElement() in the Python3 runtime when a TOKEN_REF (or string) in the path does not name a token in the lexer's rule names / literal names. The XPath evaluator must map the name to a token type to match terminal nodes; an unknown name cannot be mapped and ttype stays INVALID_TYPE.

Source

Thrown at runtime/Python3/src/antlr4/xpath/XPath.py:153

            raise Exception("Missing path element at end of path")

        word = wordToken.text
        if wordToken.type==XPathLexer.WILDCARD :
            return XPathWildcardAnywhereElement() if anywhere else XPathWildcardElement()

        elif wordToken.type in [XPathLexer.TOKEN_REF, XPathLexer.STRING]:
            tsource = self.parser.getTokenStream().tokenSource

            ttype = Token.INVALID_TYPE
            if wordToken.type == XPathLexer.TOKEN_REF:
                if word in tsource.ruleNames:
                    ttype = tsource.ruleNames.index(word) + 1
            else:
                if word in tsource.literalNames:
                    ttype = tsource.literalNames.index(word)

            if ttype == Token.INVALID_TYPE:
                raise Exception("%s at index %d isn't a valid token name" % (word, wordToken.tokenIndex))
            return XPathTokenAnywhereElement(word, ttype) if anywhere else XPathTokenElement(word, ttype)

        else:
            ruleIndex = self.parser.ruleNames.index(word) if word in self.parser.ruleNames else -1

            if ruleIndex == -1:
                raise Exception("%s at index %d isn't a valid rule name" % (word, wordToken.tokenIndex))
            return XPathRuleAnywhereElement(word, ruleIndex) if anywhere else XPathRuleElement(word, ruleIndex)


    @staticmethod
    def findAll(tree:ParseTree, xpath:str, parser:Parser):
        p = XPath(parser, xpath)
        return p.evaluate(tree)

    #
    # Return a list of all nodes starting at {@code t} as root that satisfy the
    # path. The root {@code /} is relative to the node passed to

View on GitHub (pinned to 7d5770395b)

Solutions

1. Check the name against the generated lexer's ruleNames / literalNames lists
2. If the name is a parser rule, use it in rule position (lowercase, no '#')
3. Regenerate and inspect the lexer vocabulary if the grammar recently changed

Example fix

# before: grammar defines ID, not IDENT
nodes = XPath.findAll(tree, "/IDENT", parser)

# after
nodes = XPath.findAll(tree, "/ID", parser)

known = set(l for l in lexer.ruleNames if l) | set(l for l in getattr(lexer, 'literalNames', []) if l)
if name not in known:
    raise ValueError(f'{name} is not a token name')

def is_known_token_name(name: str, lexer) -> bool:
    return name in (lexer.ruleNames or []) or name in (getattr(lexer, 'literalNames', None) or [])

try:
    nodes = XPath.findAll(tree, xpath, parser)
except Exception as e:
    if "isn't a valid token name" in str(e):
        # look up correct token name in lexer vocabulary
        ...

When it happens

Trigger: Calling XPath with a path like '/unknownName' where the first character convention marks it a token reference, or '/"literal"' whose text does not appear in the lexer's literal names.

Common situations: Grammar changes renamed or removed tokens; using a rule name in token position; string literals that do not exactly match a literal token name.

Related errors

• Missing element after %s
• Unknown path element %s
• Missing path element at end of path
• %s at index %d isn't a valid rule name
• The current recognizer does not provide a list of token name

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/33effc35bd595e90. Report an issue: GitHub.