dotnet/machinelearning · error · ArgumentException

offsetBuffer

Error message

offsetBuffer

What it means

The ArrowStringDataFrameColumn constructor validates that the offsets buffer contains exactly length + 1 entries (Arrow string-array layout requires one offset per value plus a final terminating offset). When offsets.Length != length + 1, it throws ArgumentException(nameof(offsetBuffer)), i.e. an exception whose message is the parameter name.

Source

Thrown at src/Microsoft.Data.Analysis/DataFrameColumns/ArrowStringDataFrameColumn.cs:54

        }

        /// <summary>
        /// Constructs an <see cref="ArrowStringDataFrameColumn"/> with the given <paramref name="name"/>, <paramref name="length"/> and <paramref name="nullCount"/>. The <paramref name="values"/>, <paramref name="offsets"/> and <paramref name="nullBits"/> are the contents of the column in the Arrow format.
        /// </summary>
        /// <param name="name">The name of the column.</param>
        /// <param name="values">The Arrow formatted string values in this column.</param>
        /// <param name="offsets">The Arrow formatted offsets in this column.</param>
        /// <param name="nullBits">The Arrow formatted null bits in this column.</param>
        /// <param name="length">The length of the column.</param>
        /// <param name="nullCount">The number of <see langword="null" /> values in this column.</param>
        public ArrowStringDataFrameColumn(string name, ReadOnlyMemory<byte> values, ReadOnlyMemory<byte> offsets, ReadOnlyMemory<byte> nullBits, int length, int nullCount) : base(name, length, typeof(string))
        {
            ReadOnlyDataFrameBuffer<byte> dataBuffer = new ReadOnlyDataFrameBuffer<byte>(values, values.Length);
            ReadOnlyDataFrameBuffer<int> offsetBuffer = new ReadOnlyDataFrameBuffer<int>(offsets, length + 1);
            ReadOnlyDataFrameBuffer<byte> nullBitMapBuffer = new ReadOnlyDataFrameBuffer<byte>(nullBits, nullBits.Length);

            if (length + 1 != offsetBuffer.Length)
                throw new ArgumentException(nameof(offsetBuffer));

            _dataBuffers = new List<ReadOnlyDataFrameBuffer<byte>>();
            _offsetsBuffers = new List<ReadOnlyDataFrameBuffer<int>>();
            _nullBitMapBuffers = new List<ReadOnlyDataFrameBuffer<byte>>();

            _dataBuffers.Add(dataBuffer);
            _offsetsBuffers.Add(offsetBuffer);
            _nullBitMapBuffers.Add(nullBitMapBuffer);

            _nullCount = nullCount;
        }

        private long _nullCount;

        /// <inheritdoc/>
        public override long NullCount => _nullCount;

        /// <inheritdoc/>

View on GitHub (pinned to 7b76e69cf9)

Solutions

  1. Ensure offsets.Length == length + 1 before constructing; recompute length from offsets as offsets.Length - 1.
  2. Verify the offsets array includes the final terminating offset equal to values.Length.
  3. Check that the buffer/byte arrays were sliced with the correct end indices when extracting from an Arrow IPC stream.
  4. Assert consistency: values.Length must equal offsets[length].

Example fix

// before
var col = new ArrowStringDataFrameColumn("s", values, offsets, nullBits, offsets.Length); // wrong length

// after
long length = offsets.Length - 1;
var col = new ArrowStringDataFrameColumn("s", values, offsets, nullBits, length); // offsets.Length == length + 1
Defensive patterns

Strategy: validation

Validate before calling

if (offsets.Length != length + 1)
    throw new ArgumentException($"offsets must have length + 1 = {length + 1} entries, got {offsets.Length}");

Try / catch

try { new ArrowStringDataFrameColumn(name, values, offsets, nullBits, length); }
catch (ArgumentException) { /* re-derive length = offsets.Length - 1 and retry */ }

Prevention

When it happens

Trigger: new ArrowStringDataFrameColumn(name, dataBuffer: values, offsets, nullBits, length) where the offsets array length is not exactly length + 1 (e.g. passing offsets for a different number of strings, or an off-by-one where the final terminating offset was omitted).

Common situations: Hand-building Arrow-format buffers from another format (IPC file, Parquet decode, custom serialization) with inconsistent metadata; forgetting the +1 terminating offset; using length = number of offset bytes instead of number of strings.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/35008de0f469bb90. Report an issue: GitHub.