{"record":{"id":"b65f347061fb1031","repo":"dotnet/machinelearning","slug":"length-exceeds-buffer-capacity","errorCode":null,"errorMessage":"{length} exceeds buffer capacity","messagePattern":"(.+?) exceeds buffer capacity","errorType":"validation","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.Data.Analysis/ReadOnlyDataFrameBuffer.cs","lineNumber":53,"sourceCode":"        protected static int Size = Unsafe.SizeOf<T>();\n\n        protected int Capacity => ReadOnlyBuffer.Length / Size;\n\n        public static int MaxCapacity => ArrayUtility.ArrayMaxSize / Size;\n\n        public ReadOnlySpan<T> ReadOnlySpan\n        {\n            [MethodImpl(MethodImplOptions.AggressiveInlining)]\n            get => (MemoryMarshal.Cast<byte, T>(ReadOnlyBuffer.Span)).Slice(0, Length);\n        }\n\n        public int Length { get; protected set; }\n\n        public ReadOnlyDataFrameBuffer(int length = 0)\n        {\n            if ((long)length * Size > MaxCapacity)\n            {\n                throw new ArgumentException($\"{length} exceeds buffer capacity\", nameof(length));\n            }\n            _readOnlyBuffer = new byte[length * Size];\n            Length = length;\n        }\n\n        public ReadOnlyDataFrameBuffer(ReadOnlyMemory<byte> buffer, int length)\n        {\n            _readOnlyBuffer = buffer;\n            Length = length;\n        }\n\n        internal virtual T this[int index]\n        {\n            get\n            {\n                if (index >= Length)\n                    throw new ArgumentOutOfRangeException(nameof(index));\n","sourceCodeStart":35,"sourceCodeEnd":71,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.Data.Analysis/ReadOnlyDataFrameBuffer.cs#L35-L71","documentation":"The ReadOnlyDataFrameBuffer constructor validates that length * Size (bytes per element) does not exceed MaxCapacity (int.MaxValue bytes). If you request more elements than fit in a single managed byte array, the constructor throws ArgumentException with parameter name 'length'. Buffers are backed by single arrays, so a buffer cannot span the 2GB limit.","triggerScenarios":"new ReadOnlyDataFrameBuffer<byte>(length) (or a derived buffer such as DataFrameBuffer<T>/ReadOnlyDataFrameBuffer<T>) with length*Size > ~2,147,483,591 bytes; e.g. a byte buffer with length > int.MaxValue, or a double buffer with more than ~268M elements requested in ONE buffer.","commonSituations":"Loading very large files or datasets into a single buffer instead of the column's multi-buffer layout; misreading Length as element capacity vs bytes; constructing a buffer manually with an inflated length instead of letting the column split data across buffers.","solutions":["Split the data into multiple smaller buffers/columns, each under MaxCapacity bytes (the DataFrameColumn APIs do this automatically)","Reduce the requested length so length * sizeof(T) <= int.MaxValue","Check your length units — pass element count, not byte count, and verify Size for the element type","For datasets near 2GB, avoid in-place buffer construction and stream/append via the column APIs"],"exampleFix":"// before\nvar buffer = new ReadOnlyDataFrameBuffer<double>(300_000_000); // 2.4GB > MaxCapacity -> ArgumentException\n// after\nconst int maxElements = int.MaxValue / sizeof(double); // ~268M\nvar buffer = new ReadOnlyDataFrameBuffer<double>(Math.Min(length, maxElements));\n// keep the remainder in a second buffer","handlingStrategy":"validation","validationCode":"static bool BufferLengthFits(long length, int elementSize) =>\n    length >= 0 && length * elementSize <= int.MaxValue;","typeGuard":"bool CanCreateBuffer<T>(int length) => (long)length * System.Runtime.CompilerServices.Unsafe.SizeOf<T>() <= int.MaxValue;","tryCatchPattern":"try { var buffer = new ReadOnlyDataFrameBuffer<double>(length); }\ncatch (ArgumentException ex) when (ex.ParamName == \"length\") {\n    // split into multiple buffers, each <= int.MaxValue / sizeof(double) elements\n}","preventionTips":["Always pass element counts, never byte counts, to buffer constructors","Remember MaxCapacity is per-buffer (int.MaxValue bytes); columns split data across buffers automatically","Use the DataFrameColumn append APIs instead of constructing giant buffers manually","Compute maxElements = int.MaxValue / sizeof(T) as a named constant in buffer-heavy code"],"tags":["dotnet","argument","buffer","memory-limit"],"backgroundTag":"argument-out-of-range","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}