{"record":{"id":"51309da69c4dc2d3","repo":"apache/beam","slug":"cannot-convert-beam-type-to-bigquery-type","errorCode":null,"errorMessage":"Cannot convert Beam type:  to BigQuery type.","messagePattern":"Cannot convert Beam type:  to BigQuery type\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java","lineNumber":375,"sourceCode":"  static StandardSQLTypeName toStandardSQLTypeName(FieldType fieldType) {\n    StandardSQLTypeName ret;\n    if (fieldType.getTypeName().isLogicalType()) {\n      Schema.LogicalType<?, ?> logicalType =\n          Preconditions.checkArgumentNotNull(fieldType.getLogicalType());\n      ret = BEAM_TO_BIGQUERY_LOGICAL_MAPPING.get(logicalType.getIdentifier());\n      if (ret == null) {\n        if (logicalType instanceof PassThroughLogicalType) {\n          return toStandardSQLTypeName(logicalType.getBaseType());\n        }\n        throw new IllegalArgumentException(\n            \"Cannot convert Beam logical type: \"\n                + logicalType.getIdentifier()\n                + \" to BigQuery type.\");\n      }\n    } else {\n      ret = BEAM_TO_BIGQUERY_TYPE_MAPPING.get(fieldType.getTypeName());\n      if (ret == null) {\n        throw new IllegalArgumentException(\n            \"Cannot convert Beam type: \" + fieldType.getTypeName() + \" to BigQuery type.\");\n      }\n    }\n    return ret;\n  }\n\n  /**\n   * Represents a timestamp with picosecond precision, split into seconds and picoseconds\n   * components.\n   */\n  public static class TimestampPicos {\n    final long seconds;\n    final long picoseconds;\n\n    TimestampPicos(long seconds, long picoseconds) {\n      this.seconds = seconds;\n      this.picoseconds = picoseconds;\n    }","sourceCodeStart":357,"sourceCodeEnd":393,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java#L357-L393","documentation":"In BigQueryUtils.toStandardSQLTypeName, non-logical field types are looked up in BEAM_TO_BIGQUERY_TYPE_MAPPING by TypeName. If the mapping has no entry (the Beam primitive/array/map/iterable type has no BigQuery equivalent), an IllegalArgumentException is thrown. This guards against attempting to write schema fields BigQuery IO cannot represent.","triggerScenarios":"Writing a PCollection whose schema contains a Beam type absent from BEAM_TO_BIGQUERY_TYPE_MAPPING (e.g. nested BYTES in unsupported position, DATETIME variants, unspecified/EXUPPORTED types) to a BigQuery sink.","commonSituations":"Nested collections (list<list<T>>) unsupported by BigQuery; map types with unsupported key/value types; new Beam TypeName added in a newer Beam than the mapping table; passing generic ROW without proper nested conversion.","solutions":["Flatten or simplify unsupported nested types before writing (avoid list<list<T>>; BigQuery only supports one level of nesting)","Convert unmapped fields to supported primitives (STRING/BYTES/INT64/FLOAT64/NUMERIC/TIMESTAMP/etc.) in a Select/ParDo","Upgrade Beam to pick up additions to BEAM_TO_BIGQUERY_TYPE_MAPPING","Inspect the schema (pcollection.getSchema()) and validate every field type against BigQuery-supported types before the sink"],"exampleFix":"// before\nSchema.builder().addArrayField(\"matrix\", FieldType.array(FieldType.INT64)).build(); // nested array\n// after\n// flatten: one level of array only, or encode as repeated struct / JSON string\nSchema.builder().addArrayField(\"values\", FieldType.INT64).build();\n// or\nSchema.builder().addStringField(\"matrixJson\").build(); // serialize nested structure to JSON","handlingStrategy":"validation","validationCode":"static final Set<TypeName> SUPPORTED = Set.of(TypeName.BYTE, TypeName.INT16, TypeName.INT32,\n    TypeName.INT64, TypeName.FLOAT, TypeName.DOUBLE, TypeName.DECIMAL, TypeName.STRING,\n    TypeName.DATETIME, TypeName.BOOLEAN, TypeName.BYTES, TypeName.ARRAY, TypeName.ROW);\nfor (Field f : schema.getFields()) {\n  if (!SUPPORTED.contains(f.getType().getTypeName())) {\n    throw new IllegalArgumentException(\"Unsupported Beam type for BigQuery: \" + f.getType().getTypeName());\n  }\n}","typeGuard":"boolean bigQuerySupported(FieldType t) {\n  return BEAM_TO_BIGQUERY_TYPE_MAPPING.containsKey(t.getTypeName()); // conceptually; validate recursively\n}","tryCatchPattern":"try {\n  rows.apply(BigQueryIO.writeTableRows().to(spec));\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().contains(\"Cannot convert Beam type\")) {\n    // reshape the schema (flatten nesting, serialize to STRING) and retry\n  } else throw e;\n}","preventionTips":["Avoid doubly-nested collections; BigQuery supports only one nesting level","Convert unmappable fields to STRING/JSON before the sink","Test schema-to-BigQuery conversion for every schema evolution","Keep Beam up to date so the type mapping table covers new TypeNames"],"tags":["bigquery","beam-schema","type-mapping","unsupported-type"],"backgroundTag":"type-mismatch","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}