{"record":{"id":"47a7e80879b37d04","repo":"apache/beam","slug":"write-to-cloud-bigtable-is-supported-for-flat-schema-only","errorCode":null,"errorMessage":"Write to Cloud Bigtable is supported for flat schema only.","messagePattern":"Write to Cloud Bigtable is supported for flat schema only\\.","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigtable/BigtableTable.java","lineNumber":117,"sourceCode":"        .apply(\"BigtableRowToBeamRow\", bigtableRowToRow())\n        .setRowSchema(schema);\n  }\n\n  @Override\n  public PCollection<Row> buildIOReader(\n      PBegin begin, BeamSqlTableFilter filters, List<String> fieldNames) {\n    BigtableIO.Read readTransform = readTransform();\n    if (filters instanceof BigtableFilter) {\n      BigtableFilter bigtableFilter = (BigtableFilter) filters;\n      readTransform = readTransform.withRowFilter(bigtableFilter.getFilters());\n    }\n    return readTransform.expand(begin).apply(bigtableRowToRow());\n  }\n\n  @Override\n  public POutput buildIOWriter(PCollection<Row> input) {\n    if (!useFlatSchema) {\n      throw new UnsupportedOperationException(\n          \"Write to Cloud Bigtable is supported for flat schema only.\");\n    }\n    BigtableIO.Write write =\n        BigtableIO.write().withProjectId(projectId).withInstanceId(instanceId).withTableId(tableId);\n    if (!emulatorHost.isEmpty()) {\n      write = write.withEmulator(emulatorHost);\n    }\n    return input.apply(new BeamRowToBigtableMutation(columnsMapping)).apply(write);\n  }\n\n  @Override\n  public PCollection.IsBounded isBounded() {\n    return PCollection.IsBounded.BOUNDED;\n  }\n\n  @Override\n  public BeamSqlTableFilter constructFilter(List<RexNode> filter) {\n    return new BigtableFilter(filter, schema);","sourceCodeStart":99,"sourceCodeEnd":135,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigtable/BigtableTable.java#L99-L135","documentation":"BigtableTable only supports writing to Bigtable when the table schema is flat (all columns map directly to a single column family, keyed by the 'key' field). buildIOWriter throws UnsupportedOperationException when useFlatSchema is false, because complex/nested schemas cannot be translated into Bigtable rows by this provider.","triggerScenarios":"Running an INSERT/DELETE/UPDATE (any DML that calls buildIOWriter) against a Bigtable SQL table whose schema is not flat, e.g. the schema has nested STRUCT/ARRAY fields or the columnsMapping implies a non-flat layout.","commonSituations":"Declaring a table with complex types to read nested Bigtable qualifiers, then attempting INSERT INTO it; schema changes that introduced nested fields after the table was created; trying to persist query output containing STRUCT/ARRAY columns into Bigtable.","solutions":["Write only from tables with flat schemas: keep all fields as primitive types plus the STRING 'key' field.","Flatten nested data before writing: project the query to flat columns first, e.g. INSERT INTO flat_bt SELECT key, s.a, s.b FROM ... .","If complex writes are genuinely needed, write via BigtableIO.Write directly in a pipeline instead of Beam SQL DML."],"exampleFix":"-- before (nested schema, write fails)\nCREATE EXTERNAL TABLE bt (key STRING, data STRUCT<a INT64>) TYPE 'bigtable' LOCATION '...';\nINSERT INTO bt SELECT key, STRUCT(1 AS a) FROM t;\n\n-- after (flat schema)\nCREATE EXTERNAL TABLE bt (key STRING, data_a INT64) TYPE 'bigtable' LOCATION '...';\nINSERT INTO bt SELECT key, 1 FROM t;","handlingStrategy":"validation","validationCode":"boolean flat = table.getSchema().getFields().stream()\n    .allMatch(f -> !f.getType().getTypeName().isCompositeType());\nif (!flat) {\n  throw new IllegalArgumentException(\"Flatten the schema before writing to Bigtable via Beam SQL\");\n}","typeGuard":"boolean isFlatSchema(Schema s) {\n  return s.getFields().stream()\n      .noneMatch(f -> f.getType().getTypeName().isCompositeType());\n}","tryCatchPattern":"try {\n  stmt.execute(\"INSERT INTO bt_table SELECT ... FROM ...\");\n} catch (UnsupportedOperationException e) {\n  if (e.getMessage().contains(\"flat schema only\")) {\n    // rewrite the query with a flat projection\n  }\n}","preventionTips":["Use only primitive-typed columns (plus STRING key) for tables you intend to INSERT into.","Flatten nested data in a preceding SELECT before writing to Bigtable.","Keep read-side (complex) and write-side (flat) tables separate in your SQL model."],"tags":["java","beam-sql","bigtable","unsupported-write"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}