antlr/antlr4 · error · IllegalStateException

release() called with an invalid marker.

Error message

release() called with an invalid marker.

What it means

UnbufferedTokenStream uses a strict LIFO marker stack: mark() returns -numMarkers - 1, and release() accepts only -numMarkers, the most recently issued marker. Releasing the wrong marker, releasing out of order, or releasing the same marker twice corrupts the buffer state, so the implementation rejects it immediately.

Source

Thrown at runtime/Java/src/org/antlr/v4/runtime/UnbufferedTokenStream.java:220

	 * protection against misuse where {@code seek()} is called on a mark or
	 * {@code release()} is called in the wrong order.</p>
	 */
	@Override
	public int mark() {
		if (numMarkers == 0) {
			lastTokenBufferStart = lastToken;
		}

		int mark = -numMarkers - 1;
		numMarkers++;
		return mark;
	}

	@Override
	public void release(int marker) {
		int expectedMark = -numMarkers;
		if ( marker!=expectedMark ) {
			throw new IllegalStateException("release() called with an invalid marker.");
		}

		numMarkers--;
		if ( numMarkers==0 ) { // can we release buffer?
			if (p > 0) {
				// Copy tokens[p]..tokens[n-1] to tokens[0]..tokens[(n-1)-p], reset ptrs
				// p is last valid token; move nothing if p==n as we have no valid char
				System.arraycopy(tokens, p, tokens, 0, n - p); // shift n-p tokens from p to 0
				n = n - p;
				p = 0;
			}

			lastTokenBufferStart = lastToken;
		}
	}

	@Override
	public int index() {

View on GitHub (pinned to 7d5770395b)

Solutions

1. Release markers in exactly the reverse order of mark().
2. Wrap each marked region in try-finally and release each marker exactly once.
3. Keep marker handles in a Deque/stack local to one stream rather than reusing variables.
4. Use BufferedTokenStream/CommonTokenStream if you do not need the streaming behavior and rely on arbitrary marker ordering.

Example fix

// before
int m1 = tokens.mark();
int m2 = tokens.mark();
tokens.release(m1); // invalid: expected m2 (-2)
tokens.release(m2);

// after
int m1 = tokens.mark();
try {
    int m2 = tokens.mark();
    try {
        // backtrack region
    } finally {
        tokens.release(m2);
    }
} finally {
    tokens.release(m1);
}

int marker = tokens.mark();
try {
    // marked work; release exactly this marker, exactly once
} finally {
    tokens.release(marker);
}

try {
    tokens.release(marker);
} catch (IllegalStateException e) {
    // marker stack is corrupt; reset/recreate the stream rather than releasing more markers
}

When it happens

Trigger: Calling mark() twice and releasing the first marker before the second; calling release(marker) twice; storing markers from several streams and passing the wrong integer; or copying a marker value and releasing it after its enclosing try-finally already released it.

Common situations: Custom backtracking code that nests marks, exception paths that release twice, and generic code that treats marker integers as interchangeable handles. BufferedTokenStream accepts arbitrary release values because its marks are no-ops, so code ported from it can expose this stricter behavior.

Related errors

• cannot seek to negative index {index}
• cannot consume EOF
• seek to index outside buffer: {index} not in {bufferStartInd
• Unbuffered stream cannot know its size
• interval {interval} not in token buffer window: {bufferStart

AI-assisted analysis of antlr/antlr4@7d5770395b (2026-08-14). Data as JSON: /api/errors/fb868f70661074be. Report an issue: GitHub.