dotnet/machinelearning · error · ArgumentException

Strings.SpansMultipleBuffers

Error message

Strings.SpansMultipleBuffers

What it means

ToArrowArray produces an Arrow StringArray backed by a single contiguous buffer. If the requested window [startIndex, startIndex + numberOfRows) extends past the end of the buffer containing startIndex, the library throws ArgumentException(Strings.SpansMultipleBuffers, nameof(numberOfRows)), because an Arrow StringArray cannot be built from a window spanning two buffers.

Source

Thrown at src/Microsoft.Data.Analysis/DataFrameColumns/ArrowStringDataFrameColumn.cs:357

        {
            int nullCount = 0;
            for (long i = startIndex; i < numberOfRows; i++)
            {
                if (!IsValid(i))
                    nullCount++;
            }
            return nullCount;
        }

        /// <inheritdoc/>
        protected internal override Apache.Arrow.Array ToArrowArray(long startIndex, int numberOfRows)
        {
            if (numberOfRows == 0)
                return new StringArray(numberOfRows, ArrowBuffer.Empty, ArrowBuffer.Empty, ArrowBuffer.Empty);
            int offsetsBufferIndex = GetBufferIndexContainingRowIndex(startIndex, out int indexInBuffer);
            if (numberOfRows != 0 && numberOfRows > _offsetsBuffers[offsetsBufferIndex].Length - 1 - indexInBuffer)
            {
                throw new ArgumentException(Strings.SpansMultipleBuffers, nameof(numberOfRows));
            }
            ArrowBuffer dataBuffer = new ArrowBuffer(_dataBuffers[offsetsBufferIndex].ReadOnlyBuffer);
            ArrowBuffer offsetsBuffer = new ArrowBuffer(_offsetsBuffers[offsetsBufferIndex].ReadOnlyBuffer);
            ArrowBuffer nullBuffer = new ArrowBuffer(_nullBitMapBuffers[offsetsBufferIndex].ReadOnlyBuffer);
            int nullCount = GetNullCount(indexInBuffer, numberOfRows);
            return new StringArray(numberOfRows, offsetsBuffer, dataBuffer, nullBuffer, nullCount, indexInBuffer);
        }

        protected internal override PrimitiveDataFrameColumn<long> GetSortIndices(bool ascending, bool putNullValuesLast) => throw new NotSupportedException();

        public new ArrowStringDataFrameColumn Clone(long numberOfNullsToAppend = 0)
        {
            return (ArrowStringDataFrameColumn)CloneImplementation(numberOfNullsToAppend);
        }

        public new ArrowStringDataFrameColumn Clone(DataFrameColumn mapIndices, bool invertMapIndices = false, long numberOfNullsToAppend = 0)
        {
            return (ArrowStringDataFrameColumn)CloneImplementation(mapIndices, invertMapIndices, numberOfNullsToAppend);

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Split the export into per-buffer chunks sized to fit within the buffer containing startIndex.
2. Use Clone/GetValue-based export (e.g. GetValues) instead of ToArrowArray when the data may span buffers.
3. Build the column in a single constructor call (one data buffer) if zero-copy Arrow export matters.
4. Check buffer boundaries via the internal offsets buffers before requesting a window.

Example fix

// before
var arr = col.ToArrowArray(startIndex: 0, numberOfRows: (int)col.Length); // may span buffers

// after
long remaining = (int)col.Length; long idx = 0;
while (remaining > 0)
{
    int chunk = (int)Math.Min(remaining, col.Length /* buffer-bounded chunk */);
    var arr = col.ToArrowArray(idx, chunk);
    idx += chunk; remaining -= chunk;
}

// ensure the window fits inside the buffer containing startIndex
int bufIdx = GetBufferIndexContainingRowIndex(startIndex, out int idxInBuf);
if (numberOfRows > bufLengths[bufIdx] - 1 - idxInBuf)
    throw new ArgumentException("Window spans multiple buffers");

try { var arr = col.ToArrowArray(start, count); }
catch (ArgumentException) { /* fall back to per-buffer chunked export */ }

When it happens

Trigger: Calling ToArrowArray(startIndex, numberOfRows) where numberOfRows exceeds the remaining values in startIndex's buffer: numberOfRows > _offsetsBuffers[offsetsBufferIndex].Length - 1 - indexInBuffer (and numberOfRows != 0).

Common situations: Exporting whole columns that internally consist of multiple appended buffers (each Append with > 0 free space adds buffers); batch-size loops that assume one buffer per column; columns built incrementally then converted to Arrow.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Value name '{0}' matches an existing column name
• {0} and {1} must be different
• string.Format(Strings.DuplicateColumnName, column.Name)
• Strings.DuplicateColumnName (formatted with column.Name)
• Strings.InvalidColumnName (formatted with columnName)

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/0c36c7e83c92f8ef. Report an issue: GitHub.