What causes "RECORD/STRUCT are not primitive types" in apache/beam?

Handle RECORD fields recursively via the full convertField/toGenericAvroSchema path, which creates a nested Avro record

apache/beam · error · IllegalArgumentException

RECORD/STRUCT are not primitive types

Error message

RECORD/STRUCT are not primitive types

What it means

BigQueryAvroUtils.getPrimitiveType converts a BigQuery standard SQL type name into a primitive Avro Schema.Type. RECORD/STRUCT are composite types, not primitives, so the method explicitly throws IllegalArgumentException instead of silently mis-mapping them.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryAvroUtils.java:157

        if (schema.getScale() != null) {
          logicalType =
              LogicalTypes.decimal(schema.getPrecision().intValue(), schema.getScale().intValue());
        } else if (schema.getPrecision() != null) {
          logicalType = LogicalTypes.decimal(schema.getPrecision().intValue());
        } else if (bqType.equals("NUMERIC")) {
          logicalType = LogicalTypes.decimal(38, 9);
        } else {
          // BIGNUMERIC
          logicalType = LogicalTypes.decimal(77, 38);
        }
        return logicalType.addToSchema(SchemaBuilder.builder().bytesType());
      case "GEOGRAPHY":
      case "JSON":
        return SchemaBuilder.builder().stringBuilder().prop("sqlType", bqType).endString();
      case "RECORD":
      case "STRUCT":
        // record
        throw new IllegalArgumentException("RECORD/STRUCT are not primitive types");
      case "RANGE": // TODO add support for range type
      default:
        throw new IllegalArgumentException("Unknown BigQuery type: " + bqType);
    }
  }

  /**
   * Formats BigQuery seconds-since-epoch into String matching JSON export. Thread-safe and
   * immutable.
   */
  private static final java.time.format.DateTimeFormatter DATE_TIME_FORMATTER =
      java.time.format.DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss")
          .withZone(java.time.ZoneOffset.UTC);

  /** Enum to define the precision of a timestamp since the epoch. */
  enum TimestampPrecision {
    MILLISECONDS,
    MICROSECONDS,

View on GitHub (pinned to 12126d8942)

Solutions

Handle RECORD fields recursively via the full convertField/toGenericAvroSchema path, which creates a nested Avro record
Flatten nested BigQuery fields before conversion
Use the library's schema-conversion entry points instead of calling getPrimitiveType directly

Example fix

// before
Schema primitive = BigQueryAvroUtils.getPrimitiveType("RECORD");
// after
Schema schema = toGenericAvroSchema("record", tableSchema.getFields()); // handles RECORD recursively

Defensive patterns

Strategy: try-catch

Validate before calling

if ("RECORD".equals(field.getType()) || "STRUCT".equals(field.getType())) { /* use recursive schema conversion instead */ }

Try / catch

try { toGenericAvroSchema(name, fields); } catch (IllegalArgumentException e) { if (e.getMessage().contains("not primitive")) { /* switch to recursive path */ } else throw e; }

Prevention

Never call getPrimitiveType for nested fields
Always use convertField/toGenericAvroSchema end-to-end
Flatten nested BigQuery schemas if only primitives are needed

When it happens

Trigger: Calling getPrimitiveType with bqType "RECORD" or "STRUCT" — i.e. asking for an Avro primitive schema for a nested BigQuery record field, usually via convertField on a TableSchema containing nested fields.

Common situations: Building an Avro schema from a BigQuery table schema that contains nested/repeated RECORD fields while treating every column as a primitive; often a bug in custom schema conversion code.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/afd4d4f19dd2e982. Report an issue: GitHub.