{"record":{"id":"acbaac3a43a120a6","repo":"antlr/antlr4","slug":"tokens-cannot-be-null-acbaac","errorCode":null,"errorMessage":"tokens cannot be null","messagePattern":"tokens cannot be null","errorType":"exception","errorClass":"ArgumentNullException","httpStatus":null,"severity":"error","filePath":"runtime/CSharp/src/ListTokenSource.cs","lineNumber":137,"sourceCode":"        /// <see cref=\"SourceName()\"/>\n        /// will attempt to infer the name from\n        /// the next\n        /// <see cref=\"IToken\"/>\n        /// (or the previous token if the end of the input has\n        /// been reached).\n        /// </param>\n        /// <exception>\n        /// NullPointerException\n        /// if\n        /// <paramref name=\"tokens\"/>\n        /// is\n        /// <see langword=\"null\"/>\n        /// </exception>\n        public ListTokenSource(IList<IToken> tokens, string sourceName)\n        {\n            if (tokens == null)\n            {\n                throw new ArgumentNullException(\"tokens cannot be null\");\n            }\n            this.tokens = tokens;\n            this.sourceName = sourceName;\n        }\n\n        /// <summary><inheritDoc/></summary>\n        public virtual int Column\n        {\n            get\n            {\n                if (i < tokens.Count)\n                {\n                    return tokens[i].Column;\n                }\n                else\n                {\n                    if (eofToken != null)\n                    {","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/CSharp/src/ListTokenSource.cs#L119-L155","documentation":"ListTokenSource replays a fixed list of already-created tokens (typically tokens captured from a previous lexing pass, e.g. during error recovery or re-lexing). The constructor requires the list to be non-null; an empty list is fine (it yields EOF), but null has no meaningful interpretation for a source whose whole purpose is that list.","triggerScenarios":"new ListTokenSource(null) or new ListTokenSource(tokens: null, sourceName: ...) — usually a variable that was never assigned, a filter that returned null instead of an empty list, or a capture step that was skipped.","commonSituations":"Re-lexing pipelines where the first pass failed and produced no token list; LINQ pipelines ending in FirstOrDefault() instead of ToList(); unit tests that stub the token producer as null.","solutions":["Pass a non-null list; use an empty list when no tokens were captured: tokens ?? new List<IToken>()","Make upstream token-capture code return an empty list rather than null on failure","For tests, build a small list of CommonToken objects explicitly"],"exampleFix":"// before\nIList<IToken> captured = MaybeCaptureTokens(input); // null when capture fails\nvar src = new ListTokenSource(captured, \"replay\"); // ArgumentNullException\n\n// after\nIList<IToken> captured = MaybeCaptureTokens(input) ?? new List<IToken>();\nvar src = new ListTokenSource(captured, \"replay\");","handlingStrategy":"validation","validationCode":"var source = new ListTokenSource(tokens ?? new List<IToken>(), sourceName);","typeGuard":"static bool HasTokenList(IList<IToken> t) => t != null;","tryCatchPattern":null,"preventionTips":["Make token-capture helpers return empty lists, never null","Replace FirstOrDefault() token-producer patterns with ToList() or explicit empty-list defaults"],"tags":["antlr","csharp","token-source","null-check","constructor"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}