apache/beam · error · UnsupportedOperationException

Unexpected Avro field schema type

Error message

Unexpected Avro field schema type %s for field named %s

What it means

convertRequiredField converts an Avro GenericRecord value into a TableRow JSON value for a given Avro field schema type. When the Avro schema type is one it cannot map (anything outside the handled INT/LONG/FLOAT/DOUBLE/BOOLEAN/STRING/BYTES/ENUM/RECORD etc. cases), it throws UnsupportedOperationException with the type and field name.

Solutions

1. Upgrade Beam to pick up support for the newer Avro logical type
2. Pre-transform the Avro record to convert unsupported fields to supported types
3. Inspect the field named in the message and adjust the BigQuery export to avoid that type

Example fix

// before
Schema.Field.of("x", Schema.FieldType.INT32) mapped from Avro INT with unsupported logical type
// after
Export/read with a supported type, e.g. cast the column to INTEGER/STRING in the query

for (Schema.Field f : record.getSchema().getFields()) { if (f.schema().getType() == Schema.Type.UNION || f.schema().getType() == Schema.Type.MAP) { /* transform or reject */ } }

static boolean isConvertible(Schema.Type t) { return EnumSet.of(INT, LONG, FLOAT, DOUBLE, BOOLEAN, STRING, BYTES, ENUM, RECORD).contains(t); }

try { row = convertGenericRecordToTableRow(record); } catch (UnsupportedOperationException e) { log.warn("Skipping unsupported field", e); }

When it happens

Trigger: Reading BigQuery-exported Avro files whose field schema type falls through the switch — e.g. unexpected UNION reaching the required path, or exotic logical types — during GenericRecord to TableRow conversion in BigQueryIO read.

Common situations: Avro exports from BigQuery with new/unsupported logical types, or hand-crafted Avro schemas containing types like MAP/FIXED reaching this converter.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• AvroCoder for GenericRecord requires a schema
• Can't represent as
• Converting BigQuery type '' to '' is not supported
• Field not nullable
• FieldType and AVRO schema don't have matching nullability

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3294c6be3fea2c9a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryAvroUtils.java:467

          // ideally byte[] but TableRowJsonCoder encodes as String
          return BaseEncoding.base64().encode(((ByteBuffer) v).array());
        }
      case STRING:
        // SQL types STRING, DATETIME, GEOGRAPHY, JSON
        // when not using logical type DATE, TIME too
        return v.toString();
      case ENUM:
        // SQL types STRING
        return v.toString();
      case FIXED:
        // SQL type BYTES
        // ideally byte[] but TableRowJsonCoder encodes as String
        return BaseEncoding.base64().encode(((ByteBuffer) v).array());
      case RECORD:
        // SQL types RECORD
        return convertGenericRecordToTableRow((GenericRecord) v);
      default:
        throw new UnsupportedOperationException(
            String.format("Unexpected Avro field schema type %s for field named %s", type, name));
    }
  }

  private static @Nullable Object convertNullableField(String name, Schema union, Object v) {
    // NULLABLE fields are represented as an Avro Union of the corresponding type and "null".
    verify(
        union.getType() == Schema.Type.UNION,
        "Expected Avro schema type UNION, not %s, for BigQuery NULLABLE field %s",
        union.getType(),
        name);
    List<Schema> unionTypes = union.getTypes();
    verify(
        unionTypes.size() == 2,
        "BigQuery NULLABLE field %s should be an Avro UNION of NULL and another type, not %s",
        name,
        union);

View on GitHub (pinned to 12126d8942)