apache/beam · error · IllegalArgumentException

Unsupported Beam type for ArrowFlightIO.write()…

Error message

Unsupported Beam type for ArrowFlightIO.write(): type.getTypeName()

What it means

During ArrowFlightIO.write's vector population, a Beam Schema FieldType was encountered that has no Arrow vector write path; the default branch throws IllegalArgumentException('Unsupported Beam type for ArrowFlightIO.write(): <typeName>'). The sink cannot convert that column type to Arrow.

Solutions

Identify the offending field from the message and convert it to a supported primitive type before write().
Flatten or serialize nested structures to strings/bytes.
Check supported types in ArrowFlightIO's writer switch and shape the schema accordingly.
Upgrade Beam — newer versions may have added support for the type.

Example fix

// before
schema with field 'meta' of unsupported nested type
.apply(ArrowFlightIO.write());
// after
row.set("meta", metaToJsonString(meta)); // STRING is supported
.apply(ArrowFlightIO.write());

Defensive patterns

Strategy: validation

Validate before calling

// pre-check that every schema field has a supported Arrow write path
Schema schema = pcoll.getSchema();
schema.getFields().forEach(f -> requireSupportedByArrowFlightWriter(f.getType()));

Try / catch

try {
  pcoll.apply(ArrowFlightIO.write());
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Unsupported Beam type for ArrowFlightIO.write()")) {
    log.error("Column type not supported by the Arrow writer", e);
  }
}

Prevention

Whitelist supported field types before building the sink schema
Convert nested/exotic fields to STRING or BYTES upstream
Add unit tests covering your production schema against ArrowFlightIO
Upgrade Beam when new Arrow type support is needed

When it happens

Trigger: Writing a Row whose schema contains a field type not handled by the writer's switch over type.getTypeName() (e.g., certain nested, map, or logical types).

Common situations: Schemas inferred from data with unsupported types, ROW/MAP/ARRAY combos not covered by the writer, version gaps between Beam type system and implemented Arrow writers.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/90cabadfda24533d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/arrow-flight/src/main/java/org/apache/beam/sdk/io/arrowflight/ArrowFlightIO.java:773

          break;
        case STRING:
          ((VarCharVector) vector)
              .setSafe(index, value.toString().getBytes(StandardCharsets.UTF_8));
          break;
        case BYTES:
          ((VarBinaryVector) vector).setSafe(index, (byte[]) value);
          break;
        case DATETIME:
          long millis;
          if (value instanceof org.joda.time.ReadableInstant) {
            millis = ((org.joda.time.ReadableInstant) value).getMillis();
          } else {
            millis = ((Number) value).longValue();
          }
          ((TimeStampMilliTZVector) vector).setSafe(index, millis);
          break;
        default:
          throw new IllegalArgumentException(
              "Unsupported Beam type for ArrowFlightIO.write(): " + type.getTypeName());
      }
    }

    private void closeConnection() {
      RuntimeException failure = null;
      FlightClient.ClientStreamListener currentListener = listener;
      listener = null;
      try {
        if (currentListener != null) {
          currentListener.completed();
          currentListener.getResult();
        }
      } catch (RuntimeException e) {
        failure = e;
      }

      VectorSchemaRoot currentRoot = root;

View on GitHub (pinned to 12126d8942)