{"record":{"id":"ccb3379888843456","repo":"antlr/antlr4","slug":"unbuffered-stream-cannot-know-its-size-ccb337","errorCode":null,"errorMessage":"Unbuffered stream cannot know its size","messagePattern":"Unbuffered stream cannot know its size","errorType":"exception","errorClass":"NotSupportedException","httpStatus":null,"severity":"error","filePath":"runtime/CSharp/src/UnbufferedTokenStream.cs","lineNumber":366,"sourceCode":"                }\n            }\n            p = i;\n            currentTokenIndex = index;\n            if (p == 0)\n            {\n                lastToken = lastTokenBufferStart;\n            }\n            else\n            {\n                lastToken = tokens[p - 1];\n            }\n        }\n\n        public virtual int Size\n        {\n            get\n            {\n                throw new NotSupportedException(\"Unbuffered stream cannot know its size\");\n            }\n        }\n\n        public virtual string SourceName\n        {\n            get\n            {\n                return TokenSource.SourceName;\n            }\n        }\n\n        [return: NotNull]\n        public virtual string GetText(Interval interval)\n        {\n            int bufferStartIndex = GetBufferStartIndex();\n            int bufferStopIndex = bufferStartIndex + tokens.Length - 1;\n            int start = interval.a;\n            int stop = interval.b;","sourceCodeStart":348,"sourceCodeEnd":384,"githubUrl":"https://github.com/antlr/antlr4/blob/7d5770395bb7b02eb56e7c62662cb1d7c08f42a3/runtime/CSharp/src/UnbufferedTokenStream.cs#L348-L384","documentation":"UnbufferedTokenStream.Size deliberately throws NotSupportedException because the stream lexes lazily and discards consumed tokens, so the total token count is unknowable without consuming everything. Any code path that reads the Size property on this stream type will fail.","triggerScenarios":"Accessing stream.Size on an UnbufferedTokenStream directly; passing the unbuffered stream to helper code (pretty-printers, error reporters, profilers, tree pattern machinery) that queries Size; calling APIs that preallocate based on token count.","commonSituations":"Swapping CommonTokenStream for UnbufferedTokenStream to reduce memory on huge inputs, then hitting library/utility code that assumes Size exists; unit-test harnesses that assert on token counts; migration from buffered to unbuffered streaming.","solutions":["Use CommonTokenStream instead when token count or full random access is needed","If only a count is required and input is finite, wrap a BufferedTokenStream over the same token source and read its Size","Count tokens manually by consuming to EOF on a separate pass if memory constraints still forbid buffering"],"exampleFix":"// before\nvar tokens = new UnbufferedTokenStream(lexer);\nint n = tokens.Size; // throws NotSupportedException\n\n// after\nvar tokens = new CommonTokenStream(lexer);\nint n = tokens.Size; // works: buffered stream knows its size","handlingStrategy":"type-guard","validationCode":"// C#: probe capability instead of catching\nbool knowsSize = stream is BufferedTokenStream; // CommonTokenStream derives from it\nint size = knowsSize ? ((BufferedTokenStream)stream).Size : CountByConsuming(stream);","typeGuard":"// C#\nstatic bool KnowsSize(Antlr4.Runtime.ITokenStream ts) => ts is Antlr4.Runtime.BufferedTokenStream;","tryCatchPattern":"try { int n = stream.Size; } catch (NotSupportedException) { // unbuffered stream: obtain count by a full consuming pass or switch to CommonTokenStream }","preventionTips":["Choose the stream type up front based on whether you need Size/random access","Keep utility code that reads Size away from UnbufferedTokenStream instances","Document which APIs require buffered streams when wrapping ANTLR in your own facade"],"tags":["antlr","csharp","token-stream","not-supported","streaming"],"backgroundTag":null,"analyzedSha":"7d5770395bb7b02eb56e7c62662cb1d7c08f42a3","analyzedAt":"2026-08-14T14:47:56.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}