{"record":{"id":"ed03890460a2f808","repo":"antlr/antlr4","slug":"tokensource-cannot-be-null-ed0389","errorCode":null,"errorMessage":"tokenSource cannot be null","messagePattern":"tokenSource cannot be null","errorType":"exception","errorClass":"ArgumentNullException","httpStatus":null,"severity":"error","filePath":"runtime/CSharp/src/BufferedTokenStream.cs","lineNumber":115,"sourceCode":"        /// and\n        /// <see cref=\"p\"/>\n        /// instead of calling\n        /// <see cref=\"LA(int)\"/>\n        /// .</li>\n        /// <li>\n        /// <see cref=\"Fetch(int)\"/>\n        /// : The check to prevent adding multiple EOF symbols into\n        /// <see cref=\"tokens\"/>\n        /// is trivial with this field.</li>\n        /// </ul>\n        /// </summary>\n        protected internal bool fetchedEOF;\n\n        public BufferedTokenStream(ITokenSource tokenSource)\n        {\n            if (tokenSource == null)\n            {\n                throw new ArgumentNullException(\"tokenSource cannot be null\");\n            }\n            this._tokenSource = tokenSource;\n        }\n\n        public virtual ITokenSource TokenSource\n        {\n            get\n            {\n                return _tokenSource;\n            }\n        }\n\n        public virtual int Index\n        {\n            get\n            {\n                return p;\n            }","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/CSharp/src/BufferedTokenStream.cs#L97-L133","documentation":"BufferedTokenStream's constructor requires an ITokenSource; the stream lazily pulls tokens from that source on demand and cannot function without one. The null check fails fast in the constructor rather than later during Fetch(), where a NullReferenceException would be much harder to trace back to the constructor call.","triggerScenarios":"new BufferedTokenStream(null), typically because a factory method returned null (e.g. a lexer creation function that failed silently) and its result was passed straight through.","commonSituations":"Dependency-injection containers resolving ITokenSource to null because the lexer wasn't registered; helper methods like CreateLexer(path) returning null on file-not-found instead of throwing; test code passing a mock token source that is null by default.","solutions":["Check the token source for null before constructing the stream and throw a descriptive error naming the source","Fix the upstream factory so it throws on failure instead of returning null (fail fast, no null propagation)","For tests, pass a real generated lexer or an explicit ListTokenSource over sample tokens"],"exampleFix":"// before\nITokenSource src = CreateLexerOrNull(path); // returns null on bad path\nvar stream = new BufferedTokenStream(src); // ArgumentNullException\n\n// after\nITokenSource src = CreateLexer(path) ?? throw new InvalidOperationException($\"No lexer for {path}\");\nvar stream = new BufferedTokenStream(src);","handlingStrategy":"validation","validationCode":"if (tokenSource == null)\n    throw new InvalidOperationException($\"Token source for {sourceName} was not created; check lexer factory/DI registration.\");\nvar stream = new BufferedTokenStream(tokenSource);","typeGuard":"static bool IsUsableTokenSource(ITokenSource src) => src != null && src.InputStream != null;","tryCatchPattern":null,"preventionTips":["Make token-source factories throw on failure instead of returning null","Register ITokenSource/lexer dependencies explicitly in DI containers"],"tags":["antlr","csharp","token-stream","null-check","constructor"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}