apache/seatunnel · error · CatalogException

Failed to drop BigQuery table:

Error message

Failed to drop BigQuery table: 

What it means

BigQueryCatalog.dropTable wraps exceptions from the BigQuery client's delete call in a CatalogException naming the table. TableNotExistException is deliberately re-thrown as-is so ignoreIfNotExists semantics are preserved; only unexpected failures (permissions, network, API errors) reach this message. The underlying cause from the google-cloud-bigquery client is attached.

Source

Thrown at seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java:314

            throw new CatalogException(
                    "Failed to create BigQuery table: " + tablePath.getFullName(), e);
        }
    }

    @Override
    public void dropTable(TablePath tablePath, boolean ignoreIfNotExists)
            throws TableNotExistException, CatalogException {
        TableId tableId = TableId.of(getDatasetName(tablePath), tablePath.getTableName());
        try {
            boolean deleted = bigquery.delete(tableId);
            if (!deleted && !ignoreIfNotExists) {
                throw new TableNotExistException(catalogName, tablePath);
            }
            log.info("BigQuery Table '{}' dropped successfully.", tablePath.getFullName());
        } catch (TableNotExistException e) {
            throw e;
        } catch (Exception e) {
            throw new CatalogException(
                    "Failed to drop BigQuery table: " + tablePath.getFullName(), e);
        }
    }

    @Override
    public void createDatabase(TablePath tablePath, boolean ignoreIfExists)
            throws DatabaseAlreadyExistException, CatalogException {
        String databaseName = getDatasetName(tablePath);
        if (databaseExists(databaseName)) {
            if (ignoreIfExists) {
                return;
            }
            throw new DatabaseAlreadyExistException(catalogName, databaseName);
        }
        try {
            bigquery.create(DatasetInfo.newBuilder(databaseName).build());
            log.info("BigQuery Dataset (database) '{}' created successfully.", databaseName);
        } catch (Exception e) {

View on GitHub (pinned to cf67b549a7)

Solutions

1. Inspect the wrapped cause for HTTP status: 403 -> grant `roles/bigquery.dataEditor` (includes tables.delete) on the dataset.
2. Verify the TablePath's project/dataset matches the credentials' project and that the table exists (`bq show PROJECT:dataset.table`).
3. Check network egress to bigquery.googleapis.com; retry if the cause is a transient socket/timeout error.
4. If your intent was 'drop only if exists', rely on the ignoreIfNotExists flag and ensure TableNotExistException (not CatalogException) is what surfaces for missing tables.
5. If using schema_save_mode=DROP in automated pipelines, pre-grant the SA delete permissions and audit IAM drift.

Example fix

// before: SA without delete permission
# gcloud projects add-iam-policy-binding PROJECT \
#   --member='serviceAccount:bq-sa@PROJECT.iam.gserviceaccount.com' \
#   --role='roles/bigquery.dataViewer'   # cannot delete

// after
# gcloud projects add-iam-policy-binding PROJECT \
#   --member='serviceAccount:bq-sa@PROJECT.iam.gserviceaccount.com' \
#   --role='roles/bigquery.dataEditor'   # includes tables.delete

// Java, pre-flight before dropTable
Table bqTable = bigquery.getTable(TableId.of(project, datasetName, tableName));
if (bqTable == null) {
    return; // nothing to drop; honor ignoreIfNotExists without hitting delete
}
try {
    bq.testIamPermissions(bqTable.getTableId(),
        java.util.Collections.singletonList("bigquery.tables.delete"));
} catch (BigQueryException e) {
    throw new IllegalStateException("No permission to drop " + tablePath.getFullName(), e);
}

static boolean isTransient(BigQueryException be) {
    int code = be.getCode();
    return code == 500 || code == 503 || code == 429 || be.getCause() instanceof java.io.IOException;
}

try {
    catalog.dropTable(tablePath, true);
} catch (TableNotExistException ignored) {
    // expected with ignoreIfNotExists=true
} catch (CatalogException e) {
    if (e.getCause() instanceof BigQueryException be && isTransient(be)) {
        // retry with backoff
    } else if (e.getCause() instanceof BigQueryException be && be.getCode() == 403) {
        throw new IllegalStateException("Missing bigquery.tables.delete for " + tablePath, e);
    }
    throw e;
}

When it happens

Trigger: Any exception other than the recognized TableNotExistException path while deleting the table: caller lacks bigquery.tables.delete permission on the dataset, network failure or timeout to the BigQuery API, table name resolves to a resource the client can't delete (e.g. project/dataset mismatch), or unexpected API error responses.

Common situations: schema_save_mode = DROP and the SA lacks delete permission; dropping across projects (table path project doesn't match the configured credentials' project); transient network drops during job cleanup; dataset permissions revoked between tableExists check and delete; retries hitting consistent 403.

Related errors

• Failed to list tables in dataset:
• Failed to create BigQuery table:
• Failed to open BigQueryCatalog
• Failed to list BigQuery databases (datasets)
• Failed to create BigQuery dataset:

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/82e8384f723ee2e8. Report an issue: GitHub.