apache/beam · error · IllegalArgumentException

Unknown Avro type: " + type.getType()

Error message

Unknown Avro type: " + type.getType()

What it means

typedTableFieldSchema maps an Avro Schema.Type back to a BigQuery TableFieldSchema type (the inverse of Avro generation). When the Avro type is one it cannot map back (in the default branch), it throws IllegalArgumentException, since not every Avro type has a BigQuery counterpart.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryAvroUtils.java:718

              }
            }
          }
          return fieldSchema;
        } else {
          return fieldSchema.setType("BYTES");
        }
      case ENUM:
        return fieldSchema.setType("STRING");
      case FIXED:
        return fieldSchema.setType("BYTES");
      case RECORD:
        List<TableFieldSchema> recordFields =
            type.getFields().stream()
                .map(f -> fromAvroFieldSchema(f, useAvroLogicalTypes))
                .collect(Collectors.toList());
        return new TableFieldSchema().setType("RECORD").setFields(recordFields);
      default:
        throw new IllegalArgumentException("Unknown Avro type: " + type.getType());
    }
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Remove or convert unsupported Avro types (MAP/FIXED) to records/bytes/strings in the source schema
2. Handle the type explicitly before fromAvroFieldSchema
3. Ensure the Avro schema was originally generated from a BigQuery schema so it round-trips

Example fix

// before
Schema mapField = SchemaBuilder.builder().map().values().stringType();
// after
// represent as array of records
Schema entry = SchemaBuilder.builder().record("entry").fields().requiredString("key").requiredString("value");
Schema mapField = SchemaBuilder.builder().array().items(entry);

Set<Schema.Type> bqMappable = EnumSet.of(BOOLEAN, INT, LONG, FLOAT, DOUBLE, BYTES, STRING, RECORD, ARRAY);
if (!bqMappable.contains(avroType.getType())) throw new IllegalArgumentException("Avro type " + avroType + " has no BigQuery counterpart");

When it happens

Trigger: Calling typedTableFieldSchema (via fromAvroFieldSchema) on an Avro schema containing an unmappable type — e.g. MAP, FIXED, or an unexpected UNION shape — when deriving a BigQuery TableSchema from an Avro schema.

Common situations: Users reading Avro files into BigQuery via Beam with schemas produced by other tools containing Avro-only types, or hand-written Avro schemas that don't round-trip through BigQuery.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• Unknown BigQuery type: " + bqType
• Reserved field name <field.name()> in user schema.
• Unsupported type <elementType.getType()>
• RECORD/STRUCT are not primitive types
• Unknown BigQuery Field Mode: %s

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/614643227ef94bc6. Report an issue: GitHub.