dotnet/machinelearning · error · NotImplementedException

type.ToString()

Error message

type.ToString()

What it means

PrimitiveDataFrameColumn<T>.ToArrowArray throws NotImplementedException(type.ToString()) when the column's DataType has no corresponding Arrow array constructor in the if/else chain (Boolean, SByte/Int16/Int32/Int64, Single/Double, Decimal, DateTime-converted, UInt64, UInt16, UInt8). The message is the type's ToString(), which at least names the offending type, unlike some other throws in this file.

Source

Thrown at src/Microsoft.Data.Analysis/PrimitiveDataFrameColumn.cs:192

                return new FloatArray(arrowValueBuffer, arrowNullBuffer, numberOfRows, nullCount, offset);
            else if (type == typeof(int))
                return new Int32Array(arrowValueBuffer, arrowNullBuffer, numberOfRows, nullCount, offset);
            else if (type == typeof(long))
                return new Int64Array(arrowValueBuffer, arrowNullBuffer, numberOfRows, nullCount, offset);
            else if (type == typeof(sbyte))
                return new Int8Array(arrowValueBuffer, arrowNullBuffer, numberOfRows, nullCount, offset);
            else if (type == typeof(short))
                return new Int16Array(arrowValueBuffer, arrowNullBuffer, numberOfRows, nullCount, offset);
            else if (type == typeof(uint))
                return new UInt32Array(arrowValueBuffer, arrowNullBuffer, numberOfRows, nullCount, offset);
            else if (type == typeof(ulong))
                return new UInt64Array(arrowValueBuffer, arrowNullBuffer, numberOfRows, nullCount, offset);
            else if (type == typeof(ushort))
                return new UInt16Array(arrowValueBuffer, arrowNullBuffer, numberOfRows, nullCount, offset);
            else if (type == typeof(byte))
                return new UInt8Array(arrowValueBuffer, arrowNullBuffer, numberOfRows, nullCount, offset);
            else
                throw new NotImplementedException(type.ToString());
        }

        public new IReadOnlyList<T?> this[long startIndex, int length]
        {
            get
            {
                if (startIndex >= Length)
                {
                    throw new ArgumentOutOfRangeException(nameof(startIndex));
                }
                return _columnContainer[startIndex, length];
            }
        }

        protected override IReadOnlyList<object> GetValues(long startIndex, int length)
        {
            if (startIndex >= Length)
            {

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Restrict Arrow export to columns with DataType in the supported set (bool, byte, sbyte, short, int, long, float, double, decimal, ushort, uint, ulong, DateTime)
2. Convert unsupported columns (e.g. char) to a supported type before exporting
3. Skip unsupported columns when building the Arrow record batch / schema
4. Upgrade Microsoft.Data.Analysis to a version that may cover additional Arrow types

Example fix

// before
var array = charColumn.ToArrowArray(0, (int)charColumn.Length); // NotImplementedException
// after
var intColumn = /* convert char data to PrimitiveDataFrameColumn<int> */;
var array = intColumn.ToArrowArray(0, (int)intColumn.Length);

static readonly HashSet<Type> ArrowArrayTypes = new() { typeof(bool), typeof(byte), typeof(sbyte), typeof(short), typeof(int), typeof(long), typeof(float), typeof(double), typeof(decimal), typeof(ushort), typeof(uint), typeof(ulong), typeof(DateTime) };
if (!ArrowArrayTypes.Contains(column.DataType)) throw new NotSupportedException($"Cannot convert {column.DataType} to an Arrow array");

static bool HasArrowArrayMapping(DataFrameColumn col) =>
    ArrowArrayTypes.Contains(col.DataType);

try { var arr = column.ToArrowArray(startIndex, numberOfRows); }
catch (NotImplementedException ex) { logger.LogWarning(ex, "No Arrow array mapping for {Type}", column.DataType); /* skip or convert */ }

When it happens

Trigger: Calling ToArrowArray on a PrimitiveDataFrameColumn<T> whose DataType is not one of the explicitly handled Arrow array types, e.g. char columns or custom struct element types.

Common situations: Exporting DataFrames containing char or user-defined-struct columns to Arrow IPC; library version gaps where a primitive's Arrow conversion was not implemented; generic pipeline code that accepts any PrimitiveDataFrameColumn<T>.

Related errors

• nameof(T)
• {fieldType.Name}
• The method or operation is not implemented.
• NotImplementedException
• offsetBuffer

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/9dc11134b80325d5. Report an issue: GitHub.