What causes "{capacity} exceeds buffer capacity" in dotnet/machinelearning?

Reduce the requested capacity to at or below MaxCapacity

dotnet/machinelearning · error · ArgumentException

{capacity} exceeds buffer capacity

Error message

{capacity} exceeds buffer capacity

What it means

DataFrameBuffer's constructor throws when the requested capacity exceeds MaxCapacity for the buffer type. Buffers are backed by contiguous memory, so an over-large capacity cannot be allocated within the allowed size.

Source

Thrown at src/Microsoft.Data.Analysis/DataFrameBuffer.cs:45

        }

        public Span<T> Span
        {
            [MethodImpl(MethodImplOptions.AggressiveInlining)]
            get => (MemoryMarshal.Cast<byte, T>(Buffer.Span)).Slice(0, Length);
        }

        public Span<T> RawSpan
        {
            [MethodImpl(MethodImplOptions.AggressiveInlining)]
            get => MemoryMarshal.Cast<byte, T>(Buffer.Span);
        }

        public DataFrameBuffer(int capacity = 0)
        {
            if ((long)capacity > MaxCapacity)
            {
                throw new ArgumentException($"{capacity} exceeds buffer capacity", nameof(capacity));
            }

            _memory = new byte[Math.Max(capacity, MinCapacity) * Size];
        }

        internal DataFrameBuffer(ReadOnlyMemory<byte> buffer, int length)
        {
            _memory = new byte[buffer.Length];
            buffer.CopyTo(_memory);
            Length = length;
        }

        public void Append(T value)
        {
            EnsureCapacity(1);

            RawSpan[Length] = value;
            Length++;

View on GitHub (pinned to 7b76e69cf9)

Solutions

Reduce the requested capacity to at or below MaxCapacity
Split the data into multiple buffers/chunks
Check capacity against DataFrameBuffer<T>.MaxCapacity before constructing

Example fix

// before
var buf = new DataFrameBuffer<int>(int.MaxValue);
// after
var buf = new DataFrameBuffer<int>(Math.Min(requested, 1_000_000));

Defensive patterns

Strategy: validation

Validate before calling

if (capacity > DataFrameBuffer<T>.MaxCapacity) throw new ArgumentException("Requested capacity too large.");

Try / catch

try { var buf = new DataFrameBuffer<T>(capacity); }
catch (ArgumentException ex) when (ex.ParamName == "capacity") { /* reduce capacity or chunk data */ }

Prevention

Clamp computed capacities against MaxCapacity before constructing
Avoid deriving capacity from unchecked multiplications of counts
Chunk oversized datasets into multiple buffers

When it happens

Trigger: new DataFrameBuffer<T>(capacity) where (long)capacity > MaxCapacity (src/Microsoft.Data.Analysis/DataFrameBuffer.cs:45); e.g. passing a row count or byte-size far beyond the allowed maximum.

Common situations: Computing capacity from multiplied row/column counts that overflow practical limits; misconfigured batch sizes; porting code that assumed unbounded buffers.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/6e66d4b622b85a7a. Report an issue: GitHub.