apache/beam · error · RuntimeException

Unexpected null element type!

Error message

Unexpected null element type!

What it means

Thrown by toProtoValue in AvroGenericRecordToStorageApiProto when converting an Avro ARRAY-typed field to a BigQuery Storage API protobuf message. The code reads the Avro schema's element type via avroSchema.getElementType() and throws a RuntimeException if it is null, since array elements cannot be converted without a known element schema. This indicates the Avro schema for the field is not actually an array with a resolvable element type.

Solutions

  1. Verify the Avro schema's array fields are proper array schemas (schema.getField(name).schema().getType() == ARRAY) before writing
  2. Rebuild the schema with SchemaBuilder.array().items(...) so getElementType() is non-null
  3. Inspect the specific field's schema at runtime and log avroSchema before conversion to find the mismatch
  4. Update/align your Beam and google-cloud-libraries versions — some schema handling bugs were fixed in later releases

Example fix

// before
Schema arraySchema = Schema.create(Schema.Type.ARRAY); // no element type
// after
Schema arraySchema = Schema.createArray(Schema.create(Schema.Type.STRING));
Defensive patterns

Strategy: validation

Validate before calling

// Java
for (Field f : avroSchema.getFields()) {
  if (f.schema().getType() == Schema.Type.ARRAY && f.schema().getElementType() == null) {
    throw new IllegalStateException("Array field without element type: " + f.name());
  }
}

Type guard

// Java
Schema et = avroSchema.getElementType();
if (et != null) { /* safe to map elements */ }

Try / catch

// Java
try {
  return toProtoValue(fieldDescriptor, arrayElementType, value);
} catch (RuntimeException e) {
  if (e.getMessage().contains("Unexpected null element type")) {
    throw new IllegalStateException("Malformed Avro array schema for field " + fieldDescriptor.getName(), e);
  }
  throw e;
}

Prevention

When it happens

Trigger: Processing a GenericRecord whose field is declared ARRAY in the branch logic but whose underlying Avro Schema returns null from getElementType() — e.g. a malformed or hand-constructed Avro Schema passed to write() of BigQueryIO with method=STORAGE_API_DIRECT

Common situations: Programmatically built Avro schemas (SchemaBuilder misuse), deserialization edge cases where nested schemas get stripped, or feeding data whose runtime schema doesn't match the schema used to build the BigQuery table schema.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ce816772a88fd0de. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/AvroGenericRecordToStorageApiProto.java:478

      } else {
        throw new IllegalArgumentException(
            "Received null value for non-nullable field " + fieldDescriptor.getName());
      }
    }
    return toProtoValue(fieldDescriptor, avroField.schema(), value);
  }

  private static Object toProtoValue(
      FieldDescriptor fieldDescriptor, Schema avroSchema, Object value) {
    switch (avroSchema.getType()) {
      case RECORD:
        return messageFromGenericRecord(
            fieldDescriptor.getMessageType(), (GenericRecord) value, null, -1);
      case ARRAY:
        Iterable<Object> iterable = (Iterable<Object>) value;
        @Nullable Schema arrayElementType = avroSchema.getElementType();
        if (arrayElementType == null) {
          throw new RuntimeException("Unexpected null element type!");
        }
        return StreamSupport.stream(iterable.spliterator(), false)
            .map(v -> toProtoValue(fieldDescriptor, arrayElementType, v))
            .collect(Collectors.toList());
      case UNION:
        TypeWithNullability type = TypeWithNullability.create(avroSchema);
        Preconditions.checkState(
            type.getType().getType() != Schema.Type.UNION,
            "Multiple non-null union types are not supported.");
        return toProtoValue(fieldDescriptor, type.getType(), value);
      case MAP:
        Map<CharSequence, Object> map = (Map<CharSequence, Object>) value;
        Schema valueType = Schema.create(avroSchema.getValueType().getType());
        if (valueType == null) {
          throw new RuntimeException("Unexpected null element type!");
        }

        return map.entrySet().stream()

View on GitHub (pinned to 12126d8942)