{"record":{"id":"fb868f70661074be","repo":"antlr/antlr4","slug":"release-called-with-an-invalid-marker-fb868f","errorCode":null,"errorMessage":"release() called with an invalid marker.","messagePattern":"release\\(\\) called with an invalid marker\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"runtime/Java/src/org/antlr/v4/runtime/UnbufferedTokenStream.java","lineNumber":220,"sourceCode":"\t * protection against misuse where {@code seek()} is called on a mark or\n\t * {@code release()} is called in the wrong order.</p>\n\t */\n\t@Override\n\tpublic int mark() {\n\t\tif (numMarkers == 0) {\n\t\t\tlastTokenBufferStart = lastToken;\n\t\t}\n\n\t\tint mark = -numMarkers - 1;\n\t\tnumMarkers++;\n\t\treturn mark;\n\t}\n\n\t@Override\n\tpublic void release(int marker) {\n\t\tint expectedMark = -numMarkers;\n\t\tif ( marker!=expectedMark ) {\n\t\t\tthrow new IllegalStateException(\"release() called with an invalid marker.\");\n\t\t}\n\n\t\tnumMarkers--;\n\t\tif ( numMarkers==0 ) { // can we release buffer?\n\t\t\tif (p > 0) {\n\t\t\t\t// Copy tokens[p]..tokens[n-1] to tokens[0]..tokens[(n-1)-p], reset ptrs\n\t\t\t\t// p is last valid token; move nothing if p==n as we have no valid char\n\t\t\t\tSystem.arraycopy(tokens, p, tokens, 0, n - p); // shift n-p tokens from p to 0\n\t\t\t\tn = n - p;\n\t\t\t\tp = 0;\n\t\t\t}\n\n\t\t\tlastTokenBufferStart = lastToken;\n\t\t}\n\t}\n\n\t@Override\n\tpublic int index() {","sourceCodeStart":202,"sourceCodeEnd":238,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/Java/src/org/antlr/v4/runtime/UnbufferedTokenStream.java#L202-L238","documentation":"UnbufferedTokenStream uses a strict LIFO marker stack: mark() returns -numMarkers - 1, and release() accepts only -numMarkers, the most recently issued marker. Releasing the wrong marker, releasing out of order, or releasing the same marker twice corrupts the buffer state, so the implementation rejects it immediately.","triggerScenarios":"Calling mark() twice and releasing the first marker before the second; calling release(marker) twice; storing markers from several streams and passing the wrong integer; or copying a marker value and releasing it after its enclosing try-finally already released it.","commonSituations":"Custom backtracking code that nests marks, exception paths that release twice, and generic code that treats marker integers as interchangeable handles. BufferedTokenStream accepts arbitrary release values because its marks are no-ops, so code ported from it can expose this stricter behavior.","solutions":["Release markers in exactly the reverse order of mark().","Wrap each marked region in try-finally and release each marker exactly once.","Keep marker handles in a Deque/stack local to one stream rather than reusing variables.","Use BufferedTokenStream/CommonTokenStream if you do not need the streaming behavior and rely on arbitrary marker ordering."],"exampleFix":"// before\nint m1 = tokens.mark();\nint m2 = tokens.mark();\ntokens.release(m1); // invalid: expected m2 (-2)\ntokens.release(m2);\n\n// after\nint m1 = tokens.mark();\ntry {\n    int m2 = tokens.mark();\n    try {\n        // backtrack region\n    } finally {\n        tokens.release(m2);\n    }\n} finally {\n    tokens.release(m1);\n}","handlingStrategy":"validation","validationCode":"int marker = tokens.mark();\ntry {\n    // marked work; release exactly this marker, exactly once\n} finally {\n    tokens.release(marker);\n}","typeGuard":null,"tryCatchPattern":"try {\n    tokens.release(marker);\n} catch (IllegalStateException e) {\n    // marker stack is corrupt; reset/recreate the stream rather than releasing more markers\n}","preventionTips":["Use try-finally around every mark().","For nested marks, release in reverse order.","Keep markers scoped to the stream and region that created them."],"tags":["antlr","java","token-stream","markers","backtracking"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}