{"record":{"id":"42afe71468596524","repo":"antlr/antlr4","slug":"cannot-consume-eof-42afe7","errorCode":null,"errorMessage":"cannot consume EOF","messagePattern":"cannot consume EOF","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"error","filePath":"runtime/CSharp/src/UnbufferedTokenStream.cs","lineNumber":190,"sourceCode":"        {\n            return GetText(ctx.SourceInterval);\n        }\n\n        [return: NotNull]\n        public virtual string GetText(IToken start, IToken stop)\n        {\n            if (start != null && stop != null)\n            {\n                return GetText(Interval.Of(start.TokenIndex, stop.TokenIndex));\n            }\n            throw new NotSupportedException(\"The specified start and stop symbols are not supported.\");\n        }\n\n        public virtual void Consume()\n        {\n            if (LA(1) == TokenConstants.EOF)\n            {\n                throw new InvalidOperationException(\"cannot consume EOF\");\n            }\n            // buf always has at least tokens[p==0] in this method due to ctor\n            lastToken = tokens[p];\n            // track last token for LT(-1)\n            // if we're at last token and no markers, opportunity to flush buffer\n            if (p == n - 1 && numMarkers == 0)\n            {\n                n = 0;\n                p = -1;\n                // p++ will leave this at 0\n                lastTokenBufferStart = lastToken;\n            }\n            p++;\n            currentTokenIndex++;\n            Sync(1);\n        }\n\n        /// <summary>","sourceCodeStart":172,"sourceCodeEnd":208,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/CSharp/src/UnbufferedTokenStream.cs#L172-L208","documentation":"UnbufferedTokenStream.Consume() requires a real token at the current position. If LA(1) is EOF, the token stream has reached end of input and consuming again is illegal. This mirrors the EOF guard on UnbufferedCharStream.Consume().","triggerScenarios":"A token loop calls Consume() once too often; Consume() is called after LT(1) already returned EOF; or the same UnbufferedTokenStream is reused after exhaustion.","commonSituations":"Manual iteration over tokens with inclusive bounds; parse loops that do not break on EOF; harnesses that consume Size+1 tokens; custom error recovery that continues after EOF.","solutions":["Check LT(1).Type != TokenConstants.EOF before every Consume().","Break token loops immediately when EOF is seen.","Create a new lexer/token stream for each parse rather than reusing an exhausted one."],"exampleFix":"// before\ntokens.Consume();\n\n// after\nif (tokens.LT(1).Type != TokenConstants.EOF)\n    tokens.Consume();","handlingStrategy":"validation","validationCode":"if (tokens.LT(1).Type != TokenConstants.EOF)\n    tokens.Consume();","typeGuard":null,"tryCatchPattern":"try { tokens.Consume(); }\ncatch (InvalidOperationException ex) when (ex.Message == \"cannot consume EOF\") { /* end token loop */ }","preventionTips":["Break loops on EOF before consuming.","Do not call Consume() more times than there are non-EOF tokens.","Create a fresh token stream for each parse."],"tags":["antlr","csharp","token-stream","eof","streaming"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}