{"record":{"id":"04082fa0b021626f","repo":"apache/seatunnel","slug":"failed-to-create-bigquery-table","errorCode":null,"errorMessage":"Failed to create BigQuery table: ","messagePattern":"Failed to create BigQuery table: ","errorType":"exception","errorClass":"CatalogException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java","lineNumber":296,"sourceCode":"            com.google.cloud.bigquery.TableConstraints constraints =\n                    com.google.cloud.bigquery.TableConstraints.newBuilder()\n                            .setPrimaryKey(bqPrimaryKey)\n                            .build();\n\n            tableDefinition =\n                    StandardTableDefinition.newBuilder()\n                            .setSchema(bqSchema)\n                            .setTableConstraints(constraints)\n                            .build();\n        } else {\n            tableDefinition = StandardTableDefinition.of(bqSchema);\n        }\n\n        try {\n            bigquery.create(TableInfo.of(tableId, tableDefinition));\n            log.info(\"BigQuery Table '{}' created successfully.\", tablePath.getFullName());\n        } catch (Exception e) {\n            throw new CatalogException(\n                    \"Failed to create BigQuery table: \" + tablePath.getFullName(), e);\n        }\n    }\n\n    @Override\n    public void dropTable(TablePath tablePath, boolean ignoreIfNotExists)\n            throws TableNotExistException, CatalogException {\n        TableId tableId = TableId.of(getDatasetName(tablePath), tablePath.getTableName());\n        try {\n            boolean deleted = bigquery.delete(tableId);\n            if (!deleted && !ignoreIfNotExists) {\n                throw new TableNotExistException(catalogName, tablePath);\n            }\n            log.info(\"BigQuery Table '{}' dropped successfully.\", tablePath.getFullName());\n        } catch (TableNotExistException e) {\n            throw e;\n        } catch (Exception e) {\n            throw new CatalogException(","sourceCodeStart":278,"sourceCodeEnd":314,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java#L278-L314","documentation":"BigQueryCatalog.createTable wraps exceptions from the actual bigquery.create(TableInfo.of(tableId, tableDefinition)) API call in a CatalogException naming the fully-qualified table. The schema passed local validation (including the streaming-PK check), so this failure comes from the BigQuery service: HTTP-level rejection of the create request. The real cause from the google-cloud-bigquery client is attached.","triggerScenarios":"bigquery.create() fails: table already exists when ignore rules don't apply, invalid field names/types in the generated BigQuery Schema (unsupported SeaTunnel type mapping, names with illegal chars), caller lacks bigquery.tables.create on the dataset, dataset does not exist, exceeding table limits, or network/quota errors.","commonSituations":"schema_save_mode = CREATE always and the table already exists; dataset was never created (create_database_if_not_exists not enabled); SeaTunnel types that map to unsupported BigQuery types (e.g. complex/map/array mappings) in the converted Schema; names containing characters BigQuery rejects; IAM changes removing the SA's dataEditor role.","solutions":["Read the wrapped cause: 409 'Already Exists' -> enable CREATE_SCHEMA_WHEN_NOT_EXISTS-style handling or call createTable only after tableExists check with ignoreIfNotExists semantics.","Create the dataset first: `bq mk --dataset PROJECT:dataset` or ensure the catalog config allows database creation (create_database_if_not_exists = true).","Grant `roles/bigquery.dataEditor` (tables.create permission) to the service account on the dataset.","Check the generated bqSchema: log the SeaTunnel CatalogTable columns and verify each type maps to a supported BigQuery field type; rename columns with illegal characters.","If a type is unmappable, add a transform to cast it to a supported type before the sink."],"exampleFix":"// before: creating table without its dataset\n// dataset \"analytics\" does not exist -> Failed to create BigQuery table: project.analytics.orders\n\n// after: create dataset first (or enable auto-create)\n# bq mk --dataset myproject:analytics\n// or in catalog options:\n// create_database_if_not_exists = true","handlingStrategy":"try-catch","validationCode":"// Java, pre-flight checks before createTable\nDataset dataset = bigquery.getDataset(DatasetId.of(project, datasetName));\nif (dataset == null) {\n    bigquery.create(DatasetInfo.of(project, datasetName)); // or fail with clear message\n}\nif (bigquery.getTable(TableId.of(project, datasetName, tableName)) != null) {\n    // decide: skip, or throw with ignoreIfExists semantics\n    log.info(\"Table already exists: {}.{}.{}\", project, datasetName, tableName);\n}\n// validate all columns map to supported BigQuery types\nschema.getColumns().forEach(c ->\n    checkArgument(SUPPORTED_TYPES.contains(c.getSourceType()),\n        \"Unsupported type for BigQuery: \" + c.getSourceType()));","typeGuard":"static boolean canCreateTable(BigQuery bq, String project, String dataset) {\n    Dataset d = bq.getDataset(DatasetId.of(project, dataset));\n    return d != null; // caller must additionally hold tables.create IAM permission\n}","tryCatchPattern":"try {\n    catalog.createTable(tablePath, table, ignoreIfExists);\n} catch (CatalogException e) {\n    if (e.getCause() instanceof BigQueryException be) {\n        if (be.getCode() == 409) {\n            log.info(\"Table {} already exists; skipping\", tablePath.getFullName());\n            return;\n        }\n        if (be.getCode() == 404) {\n            throw new IllegalStateException(\"Dataset missing for \" + tablePath.getFullName(), e);\n        }\n    }\n    throw e;\n}","preventionTips":["Create the dataset before the table, or enable create_database_if_not_exists","Grant roles/bigquery.dataEditor so the SA has tables.create","Use schema_save_mode=CREATE_SCHEMA_WHEN_NOT_EXISTS style settings to avoid 409s","Pre-map SeaTunnel types to BigQuery-supported types and cast unsupported ones with a transform","Verify the TablePath project matches the credentials' project"],"tags":["bigquery","gcp","create-table","permissions","schema"],"backgroundTag":"database-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}