antlr/antlr4 · error · Exception

invalid tag: %s in pattern: %s

Error message

invalid tag: %s in pattern: %s

What it means

Raised by ParseTreePatternMatcher.tokenize() when a tag's first character is neither uppercase nor lowercase, so the matcher cannot decide whether it is a token tag or a rule tag. ANTLR's convention is uppercase-first = token tag, lowercase-first = rule tag; digits, '_', or symbols as the first character are rejected.

Source

Thrown at runtime/Python3/src/antlr4/tree/ParseTreePatternMatcher.py:296

        # create token stream from text and tags
        tokens = list()
        for chunk in chunks:
            if isinstance( chunk, TagChunk ):
                # add special rule token or conjure up new token from name
                if chunk.tag[0].isupper():
                    ttype = self.parser.getTokenType(chunk.tag)
                    if ttype==Token.INVALID_TYPE:
                        raise Exception("Unknown token " + str(chunk.tag) + " in pattern: " + pattern)
                    tokens.append(TokenTagToken(chunk.tag, ttype, chunk.label))
                elif chunk.tag[0].islower():
                    ruleIndex = self.parser.getRuleIndex(chunk.tag)
                    if ruleIndex==-1:
                        raise Exception("Unknown rule " + str(chunk.tag) + " in pattern: " + pattern)
                    ruleImaginaryTokenType = self.parser.getATNWithBypassAlts().ruleToTokenType[ruleIndex]
                    tokens.append(RuleTagToken(chunk.tag, ruleImaginaryTokenType, chunk.label))
                else:
                    raise Exception("invalid tag: " + str(chunk.tag) + " in pattern: " + pattern)
            else:
                self.lexer.setInputStream(InputStream(chunk.text))
                t = self.lexer.nextToken()
                while t.type!=Token.EOF:
                    tokens.append(t)
                    t = self.lexer.nextToken()
        return tokens

    # Split {@code <ID> = <e:expr> ;} into 4 chunks for tokenizing by {@link #tokenize}.#
    def split(self, pattern:str):
        p = 0
        n = len(pattern)
        chunks = list()
        # find all start and stop indexes first, then collect
        starts = list()
        stops = list()
        while p < n :
            if p == pattern.find(self.escape + self.start, p):

View on GitHub (pinned to 7d5770395b)

Solutions

  1. Rename the tag so it starts with a letter (uppercase for tokens, lowercase for rules)
  2. Verify custom delimiters are set correctly so plain text is not mistaken for a tag
  3. Escape literal delimiter characters with the escape sequence instead of letting them form tags

Example fix

# before
pattern = parser.compilePattern('<_value>', 0)

# after
pattern = parser.compilePattern('<value>', 0)
Defensive patterns

Strategy: validation

Validate before calling

import re
assert re.match(r'^[A-Za-z]', tag), 'tag must start with a letter'

Type guard

def is_valid_tag(tag: str) -> bool:
    return len(tag) > 0 and tag[0].isalpha()

Prevention

When it happens

Trigger: Compiling a pattern with a tag like <_id>, <9num>, <id:1x>, or an empty tag <> after delimiter handling. Unicode or punctuation-first tag names also trigger it.

Common situations: Using non-identifier tag names, or a mangled tag after misconfigured delimiters cause part of the text to be read as a tag.

Related errors

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/71b3de05318354ed. Report an issue: GitHub.