{"record":{"id":"9d353e0408be516a","repo":"antlr/antlr4","slug":"cannot-consume-eof-9d353e","errorCode":null,"errorMessage":"cannot consume EOF","messagePattern":"cannot consume EOF","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"error","filePath":"runtime/CSharp/src/BufferedTokenStream.cs","lineNumber":189,"sourceCode":"                {\n                    // the last token in tokens is EOF. skip check if p indexes any\n                    // fetched token except the last.\n                    skipEofCheck = p < tokens.Count - 1;\n                }\n                else\n                {\n                    // no EOF token in tokens. skip check if p indexes a fetched token.\n                    skipEofCheck = p < tokens.Count;\n                }\n            }\n            else\n            {\n                // not yet initialized\n                skipEofCheck = false;\n            }\n            if (!skipEofCheck && LA(1) == IntStreamConstants.EOF)\n            {\n                throw new InvalidOperationException(\"cannot consume EOF\");\n            }\n            if (Sync(p + 1))\n            {\n                p = AdjustSeekIndex(p + 1);\n            }\n        }\n\n        /// <summary>\n        /// Make sure index\n        /// <paramref name=\"i\"/>\n        /// in tokens has a token.\n        /// </summary>\n        /// <returns>\n        ///\n        /// <see langword=\"true\"/>\n        /// if a token is located at index\n        /// <paramref name=\"i\"/>\n        /// , otherwise","sourceCodeStart":171,"sourceCodeEnd":207,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/CSharp/src/BufferedTokenStream.cs#L171-L207","documentation":"BufferedTokenStream.Consume() advances the stream pointer by one token, but the token stream is logically terminated by a single EOF token; consuming EOF would move the index past the end of the stream, which is meaningless. The check fires when LA(1) already returns IntStreamConstants.EOF and the skip-EofCheck optimization does not apply (i.e. the caller has not pre-fetched beyond the current position).","triggerScenarios":"Calling Consume() while the current lookahead token is EOF: a parser or manual loop that consumes one token too many; custom ITokenSource implementations that emit EOF more than once or at the wrong time (the fetchedEOF flag then disagrees with the source); calling Consume() after the parser already reported EOF as part of an error-recovery path.","commonSituations":"Hand-written token loops like while (stream.LA(1) != Eof) { ...; stream.Consume(); } that also consume inside the body; custom lexers that emit multiple EOF tokens; parsers or listeners that consume tokens themselves while ANTLR's generated code also consumes.","solutions":["Guard every manual Consume() with a check that LA(1) != IntStreamConstants.EOF","If you have a custom ITokenSource, ensure it emits exactly one EOF token and never emits tokens after EOF","Do not consume tokens from listener/visitor callbacks while the generated parser is driving the same stream"],"exampleFix":"// before\nwhile (true) { Process(stream.LT(1)); stream.Consume(); } // throws at EOF\n\n// after\nwhile (stream.LA(1) != IntStreamConstants.EOF)\n{\n    Process(stream.LT(1));\n    stream.Consume();\n}","handlingStrategy":"validation","validationCode":"if (stream.LA(1) == IntStreamConstants.EOF)\n    return; // nothing left to consume\nstream.Consume();","typeGuard":"static bool CanConsume(ITokenStream s) => s.LA(1) != IntStreamConstants.EOF;","tryCatchPattern":null,"preventionTips":["Always test LA(1) != EOF immediately before every manual Consume()","Custom ITokenSource implementations must emit exactly one EOF token, last","Don't consume the shared stream from listener callbacks while the parser drives it"],"tags":["antlr","csharp","token-stream","eof","api-misuse"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-16T03:17:38.424Z"}