{"record":{"id":"8742ecfe9eed72a3","repo":"antlr/antlr4","slug":"nexttoken-requires-a-non-null-input-stream-8742ec","errorCode":null,"errorMessage":"nextToken requires a non-null input stream.","messagePattern":"nextToken requires a non-null input stream\\.","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"error","filePath":"runtime/CSharp/src/Lexer.cs","lineNumber":145,"sourceCode":"            _hitEOF = false;\n            _mode = Antlr4.Runtime.Lexer.DEFAULT_MODE;\n            _modeStack.Clear();\n            Interpreter.Reset();\n        }\n\n        /// <summary>\n        /// Return a token from this source; i.e., match a token on the char\n        /// stream.\n        /// </summary>\n        /// <remarks>\n        /// Return a token from this source; i.e., match a token on the char\n        /// stream.\n        /// </remarks>\n        public virtual IToken NextToken()\n        {\n            if (_input == null)\n            {\n                throw new InvalidOperationException(\"nextToken requires a non-null input stream.\");\n            }\n            // Mark start location in char stream so unbuffered streams are\n            // guaranteed at least have text of current token\n            int tokenStartMarker = _input.Mark();\n            try\n            {\n                while (true)\n                {\n                    if (_hitEOF)\n                    {\n                        EmitEOF();\n                        return _token;\n                    }\n                    _token = null;\n                    _channel = TokenConstants.DefaultChannel;\n                    _tokenStartCharIndex = _input.Index;\n                    _tokenStartColumn = Interpreter.Column;\n                    _tokenStartLine = Interpreter.Line;","sourceCodeStart":127,"sourceCodeEnd":163,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/CSharp/src/Lexer.cs#L127-L163","documentation":"Lexer.NextToken() pulls characters from the ICharStream stored in _input (set by the Recognizer base's InputStream setter / constructor). If the lexer has no input stream attached, there is nothing to tokenize, so the method throws InvalidOperationException immediately rather than NRE-ing inside the matching loop. The lexer also resets per-token state (tokenStartCharIndex, mode stack) from _input, so the check is essential.","triggerScenarios":"Calling NextToken() on a lexer constructed without a stream, or after setting InputStream to null; a lexer subclass whose constructor forgets to pass the stream to base; DI frameworks creating the lexer with a parameterless constructor and never assigning InputStream.","commonSituations":"Using the parameterless constructor for test harnesses then forgetting lexer.InputStream = new AntlrInputStream(text); resetting a pooled lexer instance between requests and clearing InputStream; subclass constructors that omit the : base(input) call.","solutions":["Always construct the lexer with the input stream: new MyLexer(new AntlrInputStream(text))","If you must set it later, assign lexer.InputStream before the first NextToken() call","For pooled/reused lexers, re-assign the stream on every checkout and assert it is non-null before use"],"exampleFix":"// before\nvar lexer = new MyLexer(); // parameterless, no stream\nlexer.Reset();\nIToken t = lexer.NextToken(); // InvalidOperationException\n\n// after\nvar lexer = new MyLexer(new AntlrInputStream(sourceText));\nIToken t = lexer.NextToken();","handlingStrategy":"validation","validationCode":"if (lexer.InputStream == null)\n    throw new InvalidOperationException(\"Lexer has no input stream; assign InputStream before lexing.\");\nIToken t = lexer.NextToken();","typeGuard":"static bool IsReadyToLex(Lexer lx) => lx != null && lx.InputStream != null;","tryCatchPattern":null,"preventionTips":["Construct lexers with the stream: new MyLexer(new AntlrInputStream(text))","For pooled lexers, re-assign InputStream on every checkout and reset"],"tags":["antlr","csharp","lexer","null-check","input-stream"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}