antlr/antlr4 · critical · Exception

Couldn't identify final state of the precedence rule prefix

Error message

Couldn't identify final state of the precedence rule prefix section.

What it means

When deserializing with generateRuleBypassTransitions enabled (needed for parse tree pattern matching, XPath, and the TokensRule bypass), the deserializer wraps each left-recursive rule by locating its StarLoopEntryState-derived end state. If no state in the ATN satisfies stateIsEndStateFor(state, ruleIndex) for a precedence rule, the wrapping cannot proceed and this Exception is raised — effectively a malformed-ATN error on the left-recursion machinery.

Source

Thrown at runtime/Python3/src/antlr4/atn/ATNDeserializer.py:240

        bypassStart.endState = bypassStop
        atn.defineDecisionState(bypassStart)

        bypassStop.startState = bypassStart

        excludeTransition = None

        if atn.ruleToStartState[idx].isPrecedenceRule:
            # wrap from the beginning of the rule to the StarLoopEntryState
            endState = None
            for state in atn.states:
                if self.stateIsEndStateFor(state, idx):
                    endState = state
                    excludeTransition = state.loopBackState.transitions[0]
                    break

            if excludeTransition is None:
                raise Exception("Couldn't identify final state of the precedence rule prefix section.")

        else:

            endState = atn.ruleToStopState[idx]

        # all non-excluded transitions that currently target end state need to target blockEnd instead
        for state in atn.states:
            for transition in state.transitions:
                if transition == excludeTransition:
                    continue
                if transition.target == endState:
                    transition.target = bypassStop

        # all transitions leaving the rule start state need to leave blockStart instead
        ruleToStartState = atn.ruleToStartState[idx]
        count = len(ruleToStartState.transitions)
        while count > 0:
            bypassStart.addTransition(ruleToStartState.transitions[count-1])

View on GitHub (pinned to 7d5770395b)

Solutions

Regenerate lexer/parser with the tool version matching the runtime so left-recursive rule ATNs are well formed.
Only enable generateRuleBypassTransitions on a copy of the options (never on defaultOptions) passed to ATNDeserializer.
If you do not need pattern matching/XPath, leave the option off — the code path is then skipped entirely.

Example fix

# before
ATNDeserializationOptions.defaultOptions.generateRuleBypassTransitions = True  # also read-only error

# after
opts = ATNDeserializationOptions(ATNDeserializationOptions.defaultOptions)
opts.generateRuleBypassTransitions = True
atn = ATNDeserializer(opts).deserialize(serialized)

Defensive patterns

Strategy: try-catch

Validate before calling

opts = ATNDeserializationOptions(ATNDeserializationOptions.defaultOptions)
if opts.generateRuleBypassTransitions:  # only needed for pattern matching / XPath
    assert tool_version == runtime_version, 'regenerate grammars before using bypass transitions'

Try / catch

try:
    atn = ATNDeserializer(opts).deserialize(data)
except Exception as e:
    if 'final state of the precedence rule' in str(e):
        raise RuntimeError('left-recursive ATN malformed: regenerate with matching ANTLR tool version')
    raise

Prevention

Enable generateRuleBypassTransitions only on an options copy and only when you need pattern matching
Regenerate grammars containing left recursion with the matching tool version

When it happens

Trigger: ATNDeserializationOptions with generateRuleBypassTransitions=True deserializing an ATN whose precedence (left-recursive) rules lack a recognizable loop-back end state — corrupt data or an ATN produced by a mismatched tool version.

Common situations: Enabling bypass transitions (required by some tree-pattern/XPath use) on generated code from an incompatible tool/runtime pair; hand-modified grammars with left recursion whose generated artifacts were stale.

Related errors

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/be90bc28a0b69f8e. Report an issue: GitHub.