{"record":{"id":"82e8384f723ee2e8","repo":"apache/seatunnel","slug":"failed-to-drop-bigquery-table","errorCode":null,"errorMessage":"Failed to drop BigQuery table: ","messagePattern":"Failed to drop BigQuery table: ","errorType":"exception","errorClass":"CatalogException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java","lineNumber":314,"sourceCode":"            throw new CatalogException(\n                    \"Failed to create BigQuery table: \" + tablePath.getFullName(), e);\n        }\n    }\n\n    @Override\n    public void dropTable(TablePath tablePath, boolean ignoreIfNotExists)\n            throws TableNotExistException, CatalogException {\n        TableId tableId = TableId.of(getDatasetName(tablePath), tablePath.getTableName());\n        try {\n            boolean deleted = bigquery.delete(tableId);\n            if (!deleted && !ignoreIfNotExists) {\n                throw new TableNotExistException(catalogName, tablePath);\n            }\n            log.info(\"BigQuery Table '{}' dropped successfully.\", tablePath.getFullName());\n        } catch (TableNotExistException e) {\n            throw e;\n        } catch (Exception e) {\n            throw new CatalogException(\n                    \"Failed to drop BigQuery table: \" + tablePath.getFullName(), e);\n        }\n    }\n\n    @Override\n    public void createDatabase(TablePath tablePath, boolean ignoreIfExists)\n            throws DatabaseAlreadyExistException, CatalogException {\n        String databaseName = getDatasetName(tablePath);\n        if (databaseExists(databaseName)) {\n            if (ignoreIfExists) {\n                return;\n            }\n            throw new DatabaseAlreadyExistException(catalogName, databaseName);\n        }\n        try {\n            bigquery.create(DatasetInfo.newBuilder(databaseName).build());\n            log.info(\"BigQuery Dataset (database) '{}' created successfully.\", databaseName);\n        } catch (Exception e) {","sourceCodeStart":296,"sourceCodeEnd":332,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java#L296-L332","documentation":"BigQueryCatalog.dropTable wraps exceptions from the BigQuery client's delete call in a CatalogException naming the table. TableNotExistException is deliberately re-thrown as-is so ignoreIfNotExists semantics are preserved; only unexpected failures (permissions, network, API errors) reach this message. The underlying cause from the google-cloud-bigquery client is attached.","triggerScenarios":"Any exception other than the recognized TableNotExistException path while deleting the table: caller lacks bigquery.tables.delete permission on the dataset, network failure or timeout to the BigQuery API, table name resolves to a resource the client can't delete (e.g. project/dataset mismatch), or unexpected API error responses.","commonSituations":"schema_save_mode = DROP and the SA lacks delete permission; dropping across projects (table path project doesn't match the configured credentials' project); transient network drops during job cleanup; dataset permissions revoked between tableExists check and delete; retries hitting consistent 403.","solutions":["Inspect the wrapped cause for HTTP status: 403 -> grant `roles/bigquery.dataEditor` (includes tables.delete) on the dataset.","Verify the TablePath's project/dataset matches the credentials' project and that the table exists (`bq show PROJECT:dataset.table`).","Check network egress to bigquery.googleapis.com; retry if the cause is a transient socket/timeout error.","If your intent was 'drop only if exists', rely on the ignoreIfNotExists flag and ensure TableNotExistException (not CatalogException) is what surfaces for missing tables.","If using schema_save_mode=DROP in automated pipelines, pre-grant the SA delete permissions and audit IAM drift."],"exampleFix":"// before: SA without delete permission\n# gcloud projects add-iam-policy-binding PROJECT \\\n#   --member='serviceAccount:bq-sa@PROJECT.iam.gserviceaccount.com' \\\n#   --role='roles/bigquery.dataViewer'   # cannot delete\n\n// after\n# gcloud projects add-iam-policy-binding PROJECT \\\n#   --member='serviceAccount:bq-sa@PROJECT.iam.gserviceaccount.com' \\\n#   --role='roles/bigquery.dataEditor'   # includes tables.delete","handlingStrategy":"try-catch","validationCode":"// Java, pre-flight before dropTable\nTable bqTable = bigquery.getTable(TableId.of(project, datasetName, tableName));\nif (bqTable == null) {\n    return; // nothing to drop; honor ignoreIfNotExists without hitting delete\n}\ntry {\n    bq.testIamPermissions(bqTable.getTableId(),\n        java.util.Collections.singletonList(\"bigquery.tables.delete\"));\n} catch (BigQueryException e) {\n    throw new IllegalStateException(\"No permission to drop \" + tablePath.getFullName(), e);\n}","typeGuard":"static boolean isTransient(BigQueryException be) {\n    int code = be.getCode();\n    return code == 500 || code == 503 || code == 429 || be.getCause() instanceof java.io.IOException;\n}","tryCatchPattern":"try {\n    catalog.dropTable(tablePath, true);\n} catch (TableNotExistException ignored) {\n    // expected with ignoreIfNotExists=true\n} catch (CatalogException e) {\n    if (e.getCause() instanceof BigQueryException be && isTransient(be)) {\n        // retry with backoff\n    } else if (e.getCause() instanceof BigQueryException be && be.getCode() == 403) {\n        throw new IllegalStateException(\"Missing bigquery.tables.delete for \" + tablePath, e);\n    }\n    throw e;\n}","preventionTips":["Ensure schema_save_mode=DROP pipelines run with an SA holding dataEditor","Check bqTable existence first so absence is handled via TableNotExistException, not delete failures","Keep TablePath project/dataset consistent with the credentials' project","Retry transient (429/5xx/IO) failures with backoff instead of failing cleanup","Audit IAM regularly — permission revocation between existence checks and deletes causes 403s"],"tags":["bigquery","gcp","drop-table","permissions","api"],"backgroundTag":"database-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}