dotnet/machinelearning · error · NotSupportedException

null

Error message

null

What it means

GetSortIndices on ArrowStringDataFrameColumn unconditionally throws NotSupportedException (no message argument, hence 'null'). Sorting is not implemented for Arrow string columns because their multi-buffer, variable-length layout has no in-place sort path. Callers reach it via DataFrame.OrderBy/Sort APIs that rely on GetSortIndices.

Source

Thrown at src/Microsoft.Data.Analysis/DataFrameColumns/ArrowStringDataFrameColumn.cs:366

        /// <inheritdoc/>
        protected internal override Apache.Arrow.Array ToArrowArray(long startIndex, int numberOfRows)
        {
            if (numberOfRows == 0)
                return new StringArray(numberOfRows, ArrowBuffer.Empty, ArrowBuffer.Empty, ArrowBuffer.Empty);
            int offsetsBufferIndex = GetBufferIndexContainingRowIndex(startIndex, out int indexInBuffer);
            if (numberOfRows != 0 && numberOfRows > _offsetsBuffers[offsetsBufferIndex].Length - 1 - indexInBuffer)
            {
                throw new ArgumentException(Strings.SpansMultipleBuffers, nameof(numberOfRows));
            }
            ArrowBuffer dataBuffer = new ArrowBuffer(_dataBuffers[offsetsBufferIndex].ReadOnlyBuffer);
            ArrowBuffer offsetsBuffer = new ArrowBuffer(_offsetsBuffers[offsetsBufferIndex].ReadOnlyBuffer);
            ArrowBuffer nullBuffer = new ArrowBuffer(_nullBitMapBuffers[offsetsBufferIndex].ReadOnlyBuffer);
            int nullCount = GetNullCount(indexInBuffer, numberOfRows);
            return new StringArray(numberOfRows, offsetsBuffer, dataBuffer, nullBuffer, nullCount, indexInBuffer);
        }

        protected internal override PrimitiveDataFrameColumn<long> GetSortIndices(bool ascending, bool putNullValuesLast) => throw new NotSupportedException();

        public new ArrowStringDataFrameColumn Clone(long numberOfNullsToAppend = 0)
        {
            return (ArrowStringDataFrameColumn)CloneImplementation(numberOfNullsToAppend);
        }

        public new ArrowStringDataFrameColumn Clone(DataFrameColumn mapIndices, bool invertMapIndices = false, long numberOfNullsToAppend = 0)
        {
            return (ArrowStringDataFrameColumn)CloneImplementation(mapIndices, invertMapIndices, numberOfNullsToAppend);
        }

        /// <inheritdoc/>
        protected override DataFrameColumn CloneImplementation(long numberOfNullsToAppend)
        {
            var ret = new ArrowStringDataFrameColumn(Name);

            for (long i = 0; i < Length; i++)
                ret.Append(IsValid(i) ? GetBytes(i) : default(ReadOnlySpan<byte>));

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Convert the ArrowStringDataFrameColumn to a StringDataFrameColumn and replace it in the collection before sorting.
2. Sort on a derived numeric column (e.g. a hash/key) instead of the Arrow string column.
3. Perform the sort upstream (in the Arrow/Feather source) before loading.
4. Clone the dataframe with mutable columns: df.Columns[name] = new StringDataFrameColumn(name, arrowCol); then OrderBy.

Example fix

// before
var sorted = df.OrderBy("Name"); // Name is ArrowStringDataFrameColumn -> NotSupportedException

// after
df["Name"] = new StringDataFrameColumn("Name", ((ArrowStringDataFrameColumn)df.Columns["Name"]).Cast<StringDataFrameColumn>());
var sorted = df.OrderBy("Name");

if (df.Columns[sortKey] is ArrowStringDataFrameColumn)
    df[sortKey] = /* convert to StringDataFrameColumn */;

static bool IsSortable(DataFrameColumn c) => c is not ArrowStringDataFrameColumn;

try { var sorted = df.OrderBy(key); }
catch (NotSupportedException) { /* convert Arrow string column to StringDataFrameColumn, retry */ }

When it happens

Trigger: Sorting a DataFrame (df.OrderBy(columnName) / DataFrameRow sorting) where the sort key column is an ArrowStringDataFrameColumn, or calling GetSortIndices directly.

Common situations: Data loaded from Arrow IPC/Feather then sorted by a string column; pipelines mixing Arrow-backed and primitive columns where sorting works for numerics but not Arrow strings.

Related errors

• Strings.ImmutableColumn
• TKey
• MismatchedColumnLengths
• Parameter must not be null, empty, or whitespace
• Value cannot be null. (Parameter 'idColumns')

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/8694da9cb8cd61cc. Report an issue: GitHub.