What causes "Type '${type}' not supported." in apache/beam?

Cast or drop the null-typed column before conversion (e.g. pyarrow cast to a concrete type)

apache/beam · error · IllegalArgumentException

Type '${type}' not supported.

Error message

Type '${type}' not supported.

What it means

During Arrow-to-Beam field conversion, the ArrowTypeVisitor's visit(ArrowType.Null) rejects Arrow null types outright — Beam Schema has no direct equivalent, so conversion throws with the Arrow type string in the message.

Source

Thrown at sdks/java/extensions/arrow/src/main/java/org/apache/beam/sdk/extensions/arrow/ArrowConversion.java:130

  /** Get Beam Field from Arrow Field. */
  private static Field toBeamField(org.apache.arrow.vector.types.pojo.Field field) {
    FieldType beamFieldType = toFieldType(field.getFieldType(), field.getChildren());
    return Field.of(field.getName(), beamFieldType);
  }

  /** Converts Arrow FieldType to Beam FieldType. */
  private static FieldType toFieldType(
      org.apache.arrow.vector.types.pojo.FieldType arrowFieldType,
      List<org.apache.arrow.vector.types.pojo.Field> childrenFields) {
    FieldType fieldType =
        arrowFieldType
            .getType()
            .accept(
                new ArrowType.ArrowTypeVisitor<FieldType>() {
                  @Override
                  public FieldType visit(ArrowType.Null type) {
                    throw new IllegalArgumentException(
                        "Type \'" + type.toString() + "\' not supported.");
                  }

                  @Override
                  public FieldType visit(ArrowType.Struct type) {
                    return FieldType.row(ArrowSchemaTranslator.toBeamSchema(childrenFields));
                  }

                  @Override
                  public FieldType visit(ArrowType.List type) {
                    checkArgument(
                        childrenFields.size() == 1,
                        "Encountered "
                            + childrenFields.size()
                            + " child fields for list type, expected 1");
                    return FieldType.array(toBeamField(childrenFields.get(0)).getType());
                  }

View on GitHub (pinned to 12126d8942)

Solutions

Cast or drop the null-typed column before conversion (e.g. pyarrow cast to a concrete type)
Pre-scan the Arrow schema for ArrowType.Null and remove those fields
Replace the field with a nullable string/int field in the Arrow schema
Catch IllegalArgumentException and treat the column as nullable string if needed

Example fix

// before
Schema beam = ArrowConversion.ArrowSchemaTranslator.toBeamSchema(arrowSchema); // has null-type col
// after (Python side)
table = table.set_column(i, table.schema.field('c').cast(pa.string()))

Defensive patterns

Strategy: validation

Validate before calling

boolean hasNullType = java.util.stream.IntStream.range(0, arrowSchema.getFields().size())
    .anyMatch(i -> arrowSchema.getFields().get(i).getType().getTypeID()
        == ArrowType.Null.INSTANCE.getTypeID());

Try / catch

try { beamSchema = toBeamSchema(arrowSchema); } catch (IllegalArgumentException e) { arrowSchema = removeNullColumns(arrowSchema); beamSchema = toBeamSchema(arrowSchema); }

Prevention

Cast all-null columns to a concrete type at the producer
Validate Arrow schemas before sending cross-language
Prefer explicit schemas over inference in pyarrow

When it happens

Trigger: toBeamSchema/toBeamField called on an Arrow schema that contains a field of ArrowType.Null (columns of all-null with null type, common from CSV/pandas inference).

Common situations: Ingesting Arrow data produced by pandas/pyarrow where a fully-null column was inferred as null type; Parquet files with null-typed columns.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4f97be7caef39a21. Report an issue: GitHub.