antlr/antlr4 · error · UnsupportedOperationException

The current recognizer does not provide a list of token name

Error message

The current recognizer does not provide a list of token names.

What it means

Recognizer.getTokenTypeMap() builds a token-name-to-type dictionary (required by XPath and tree-pattern compilation) and raises UnsupportedOperationException when getTokenNames() returns None. Generated parsers populate tokenNames; hand-written or partially configured recognizers leave it None, and this guard fires the first time a name-based lookup is attempted.

Source

Thrown at runtime/Python3/src/antlr4/Recognizer.py:56

        rvmajor, rvminor = self.extractVersion(runtimeVersion)
        tvmajor, tvminor = self.extractVersion(toolVersion)
        if rvmajor!=tvmajor or rvminor!=tvminor:
            print("ANTLR runtime and generated code versions disagree: "+runtimeVersion+"!="+toolVersion)

    def addErrorListener(self, listener):
        self._listeners.append(listener)

    def removeErrorListener(self, listener):
        self._listeners.remove(listener)

    def removeErrorListeners(self):
        self._listeners = []

    def getTokenTypeMap(self):
        tokenNames = self.getTokenNames()
        if tokenNames is None:
            from .error.Errors import UnsupportedOperationException
            raise UnsupportedOperationException("The current recognizer does not provide a list of token names.")
        result = self.tokenTypeMapCache.get(tokenNames, None)
        if result is None:
            result = zip( tokenNames, range(0, len(tokenNames)))
            result["EOF"] = Token.EOF
            self.tokenTypeMapCache[tokenNames] = result
        return result

    # Get a map from rule names to rule indexes.
    #
    # <p>Used for XPath and tree pattern compilation.</p>
    #
    def getRuleIndexMap(self):
        ruleNames = self.getRuleNames()
        if ruleNames is None:
            from .error.Errors import UnsupportedOperationException
            raise UnsupportedOperationException("The current recognizer does not provide a list of rule names.")
        result = self.ruleIndexMapCache.get(ruleNames, None)
        if result is None:

View on GitHub (pinned to 7d5770395b)

Solutions

1. Use the tool-generated recognizer class, which sets tokenNames from the grammar's token vocabulary
2. For custom recognizers, assign self.tokenNames = [...] (ordered by token type) before calling name-based APIs
3. Verify tool/runtime version alignment if generated code suddenly lacks tokenNames after an upgrade

Example fix

# before
class MyParser(Parser):
    tokenNames = None  # getTokenTypeMap() raises

# after
class MyParser(Parser):
    tokenNames = ["<INVALID>", "ID", "INT", "PLUS", "EOF"]  # indexed by token type

# Python: check names exist before name-based APIs
if recognizer.getTokenNames() is None:
    raise ValueError("recognizer has no token names; use a generated parser")
ttype = recognizer.getTokenTypeMap()["ID"]

# Python
def has_token_names(recognizer) -> bool:
    return recognizer.getTokenNames() is not None

try:
    m = recognizer.getTokenTypeMap()
except UnsupportedOperationException as ex:
    if "list of token names" in str(ex):
        # regenerate the recognizer or populate tokenNames before retry
        raise
    raise

When it happens

Trigger: Calling getTokenTypeMap(), XPath.evaluate(), or compileParseTreePattern() on a Parser/Recognizer whose tokenNames attribute is None; constructing a bare Recognizer subclass without generated metadata; custom parser wrappers that drop the generated fields.

Common situations: Hand-written recognizers used with ANTLR tree utilities; runtime/generated-code version mismatch where the tokenNames field name or initialization changed; refactors that bypass generated __init__ paths.

Related errors

• The current recognizer does not provide a list of rule names
• {word} at index {startIndex} isn't a valid token name
• Invalid tokens or characters at index %d in path '%s'
• Missing element after %s
• Unknown path element %s

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/7b63a8a57b0d0c8e. Report an issue: GitHub.