dotnet/machinelearning · error · NotImplementedException

{fieldType.Name}

Error message

{fieldType.Name}

What it means

FromArrowRecordBatch / AppendDataFrameColumnFromArrowArray throw NotImplementedException when an Arrow record batch contains a field whose ArrowTypeId has no mapping to a DataFrameColumn. Unsupported type ids include Map, Null, Time32, Time64 and others that hit the switch's default branch.

Source

Thrown at src/Microsoft.Data.Analysis/DataFrame.Arrow.cs:141

                        dataFrameColumn = dataTimeDataFrameColumn;
                    }
                    break;
                case ArrowTypeId.Decimal128:
                case ArrowTypeId.Decimal256:
                case ArrowTypeId.Binary:
                case ArrowTypeId.Date32:
                case ArrowTypeId.Dictionary:
                case ArrowTypeId.FixedSizedBinary:
                case ArrowTypeId.HalfFloat:
                case ArrowTypeId.Interval:
                case ArrowTypeId.List:
                case ArrowTypeId.Map:
                case ArrowTypeId.Null:
                case ArrowTypeId.Time32:
                case ArrowTypeId.Time64:

                default:
                    throw new NotImplementedException($"{fieldType.Name}");
            }

            if (dataFrameColumn != null)
            {
                ret.Columns.Insert(ret.Columns.Count, dataFrameColumn);
            }
        }

        /// <summary>
        /// Wraps a <see cref="DataFrame"/> around an Arrow <see cref="RecordBatch"/> without copying data
        /// </summary>
        /// <param name="recordBatch"></param>
        /// <returns><see cref="DataFrame"/></returns>
        public static DataFrame FromArrowRecordBatch(RecordBatch recordBatch)
        {
            DataFrame ret = new DataFrame();
            Apache.Arrow.Schema arrowSchema = recordBatch.Schema;
            int fieldIndex = 0;

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Preprocess the Arrow data upstream so unsupported fields (Map, Null, Time32, Time64) are converted to supported types (e.g. cast Time32/Time64 to Timestamp or Int64, Map to List/Struct).
2. Filter unsupported columns out of the Arrow schema before calling FromArrowRecordBatch.
3. Convert the batch to IPC/records yourself and build columns manually with DataFrameColumn.Create for unsupported types.
4. Check the Apache.Arrow version to see if newer mappings were added, and upgrade the package.

Example fix

// before
var df = DataFrame.FromArrowRecordBatch(batch); // batch has Time32 column -> NotImplementedException
// after
var supportedSchema = batch.Schema.RemoveField(batch.Schema.GetFieldIndex("myTime"));
var df = DataFrame.FromArrowRecordBatch(new Apache.Arrow.RecordBatch(supportedSchema, /* supported arrays only */));

var unsupported = batch.Schema.Fields
    .Where(f => f.DataType.Type is ArrowTypeId.Map or ArrowTypeId.Null or ArrowTypeId.Time32 or ArrowTypeId.Time64)
    .Select(f => f.Name)
    .ToList();
if (unsupported.Count > 0) throw new InvalidOperationException($"Unsupported Arrow columns: {string.Join(',', unsupported)}");

static bool IsArrowTypeSupported(ArrowTypeId t) =>
    t is not (ArrowTypeId.Map or ArrowTypeId.Null or ArrowTypeId.Time32 or ArrowTypeId.Time64);

try
{
    var df = DataFrame.FromArrowRecordBatch(batch);
}
catch (NotImplementedException ex)
{
    // ex.Message contains the Arrow field type name; convert/filter that column and retry
}

When it happens

Trigger: Calling DataFrame.FromArrowRecordBatch on a record batch whose schema contains Arrow columns of type Map, Null, Time32, Time64, or any other type not handled by the switch in AppendDataFrameColumnFromArrowArray.

Common situations: Reading Arrow data produced by systems that emit time32/time64 or map columns (Spark, Polars, Pandas exports); schema changes upstream introducing new field types; round-tripping data types that DataFrame does not model.

Related errors

• nameof(T)
• type.ToString()
• The method or operation is not implemented.
• NotImplementedException
• offsetBuffer

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/277e0169a13ddb2b. Report an issue: GitHub.