apache/beam · error · IllegalArgumentException

Arrow schema conversion does not support Beam type '%s' for

Error message

Arrow schema conversion does not support Beam type '%s' for field '%s'.

What it means

toArrowField maps Beam schema field types to Arrow types; Beam types with no Arrow equivalent (e.g. ITERABLE, MAP, logical types not handled) hit the default branch and throw. The library intentionally limits conversion to primitive-compatible types.

Source

Thrown at sdks/java/extensions/arrow/src/main/java/org/apache/beam/sdk/extensions/arrow/ArrowConversion.java:99

        arrowType = new ArrowType.FloatingPoint(FloatingPointPrecision.SINGLE);
        break;
      case DOUBLE:
        arrowType = new ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE);
        break;
      case STRING:
        arrowType = ArrowType.Utf8.INSTANCE;
        break;
      case BOOLEAN:
        arrowType = ArrowType.Bool.INSTANCE;
        break;
      case BYTES:
        arrowType = ArrowType.Binary.INSTANCE;
        break;
      case DATETIME:
        arrowType = new ArrowType.Timestamp(TimeUnit.MILLISECOND, "UTC");
        break;
      default:
        throw new IllegalArgumentException(
            String.format(
                "Arrow schema conversion does not support Beam type '%s' for field '%s'.",
                beamFieldType.getTypeName(), field.getName()));
    }

    org.apache.arrow.vector.types.pojo.FieldType arrowFieldType =
        beamFieldType.getNullable()
            ? org.apache.arrow.vector.types.pojo.FieldType.nullable(arrowType)
            : org.apache.arrow.vector.types.pojo.FieldType.notNullable(arrowType);
    return new org.apache.arrow.vector.types.pojo.Field(
        field.getName(), arrowFieldType, Collections.emptyList());
  }

  /** Get Beam Field from Arrow Field. */
  private static Field toBeamField(org.apache.arrow.vector.types.pojo.Field field) {
    FieldType beamFieldType = toFieldType(field.getFieldType(), field.getChildren());
    return Field.of(field.getName(), beamFieldType);
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Flatten or drop unsupported fields from the Beam Schema before conversion
2. Map unsupported types manually (e.g. convert ITERABLE to repeated primitives, DECIMAL to string) before calling toArrowField
3. Extend the switch in ArrowConversion to add the needed mapping and contribute upstream
4. Catch IllegalArgumentException and skip/convert the offending field by name reported in the message

Example fix

// before
Schema schema = row.getSchema(); // contains ITERABLE field
VectorSchemaRoot root = ArrowConversion.toArrowVector(allocator, rows, schema);
// after
Schema flat = new Schema(schema.getFields().stream()
    .filter(f -> f.getType().getTypeName().isPrimitiveType())
    .collect(Collectors.toList()));
VectorSchemaRoot root = ArrowConversion.toArrowVector(allocator, rows, flat);

java.util.Set<TypeName> supported = Set.of(TypeName.BYTE, TypeName.INT16, TypeName.INT32,
    TypeName.INT64, TypeName.FLOAT, TypeName.DOUBLE, TypeName.STRING, TypeName.BOOLEAN,
    TypeName.DATETIME);
for (Field f : schema.getFields()) {
  if (!supported.contains(f.getType().getTypeName())) {
    throw new IllegalArgumentException("Unsupported field for Arrow: " + f.getName());
  }
}

try { root = ArrowConversion.toArrowVector(allocator, rows, schema); } catch (IllegalArgumentException e) { schema = dropUnsupportedFields(schema); root = ArrowConversion.toArrowVector(allocator, rows, schema); }

When it happens

Trigger: Converting a Beam Schema to Arrow via ArrowConversion when the Schema contains a field whose FieldType has an unsupported TypeName (anything not covered by the switch: beyond BYTE..DATETIME, e.g. ITERABLE, MAP, DECIMAL, logical types).

Common situations: Converting rows from sources with nested or logical types (e.g. JsonToRow schemas, BigQuery logical types) to Arrow record batches for cross-language/instance output.

Related errors

• Type '${type}' not supported.
• Unsupported type: {fieldType}
• Error processing struct to row: {e.getMessage()}
• Unrecognized Iceberg Type: {typeId}
• Cannot provide a coder for a Beam Row. Please provide a sche

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1f05c5c7829e0184. Report an issue: GitHub.