apache/seatunnel · error · CatalogException

Failed to drop BigQuery dataset:

Error message

Failed to drop BigQuery dataset: 

What it means

This CatalogException is thrown by BigQueryCatalog.dropDatabase when bigquery.delete(databaseName, DatasetDeleteOption.deleteContents()) fails. The call attempts to delete a BigQuery dataset along with all of its tables; the exception wraps the underlying API error (permission, not found, concurrent modification, network) while naming the dataset.

Source

Thrown at seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java:351

            throw new CatalogException("Failed to create BigQuery dataset: " + databaseName, e);
        }
    }

    @Override
    public void dropDatabase(TablePath tablePath, boolean ignoreIfNotExists)
            throws DatabaseNotExistException, CatalogException {
        String databaseName = getDatasetName(tablePath);
        if (!databaseExists(databaseName)) {
            if (ignoreIfNotExists) {
                return;
            }
            throw new DatabaseNotExistException(catalogName, databaseName);
        }
        try {
            bigquery.delete(databaseName, BigQuery.DatasetDeleteOption.deleteContents());
            log.info("BigQuery Dataset (database) '{}' dropped successfully.", databaseName);
        } catch (Exception e) {
            throw new CatalogException("Failed to drop BigQuery dataset: " + databaseName, e);
        }
    }

    @Override
    public void truncateTable(TablePath tablePath, boolean ignoreIfNotExists)
            throws TableNotExistException, CatalogException {
        if (!tableExists(tablePath)) {
            if (ignoreIfNotExists) {
                return;
            }
            throw new TableNotExistException(catalogName, tablePath);
        }
        String query =
                String.format(
                        "TRUNCATE TABLE `%s.%s` ;",
                        getDatasetName(tablePath), tablePath.getTableName());
        QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query).build();
        try {

View on GitHub (pinned to cf67b549a7)

Solutions

1. Inspect the wrapped cause for the real BigQuery error (404 not found, 403 permission)
2. Verify the service account has bigquery.datasets.delete (e.g. roles/bigquery.dataOwner)
3. Re-check that the dataset still exists and the configured project/dataset names are correct; pass ignoreIfNotExists=true if races are expected and tolerate 404
4. Retry on transient 429/5xx errors

Example fix

// before
catalog.dropDatabase(tablePath, false);
// after
if (catalog.databaseExists(tablePath)) {
    catalog.dropDatabase(tablePath, true);
}

if (catalog.databaseExists(tablePath)) {
    catalog.dropDatabase(tablePath, true);
}

try {
    catalog.dropDatabase(tablePath, true);
} catch (CatalogException e) {
    if (e.getCause() instanceof BigQueryException && ((BigQueryException) e.getCause()).getCode() == 404) {
        LOG.warn("Dataset already gone, ignoring");
    } else {
        throw e;
    }
}

When it happens

Trigger: Calling dropDatabase(tablePath, ignoreIfNotExists) when the service account lacks bigquery.datasets.delete permission, the dataset was deleted concurrently (404), the BigQuery API returns an unexpected error, or credentials/network fail mid-call.

Common situations: Dataset removed by another job/user between the existence check and the delete; read-only or viewer-role service account; dangling dataset references after a failed job; GCP project switched in config so the dataset does not exist there.

Related errors

• Failed to create BigQuery dataset:
• Failed to open BigQueryCatalog
• Failed to list BigQuery databases (datasets)
• Failed to list tables in dataset:
• Failed to create BigQuery table:

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/d671d3188d0adc83. Report an issue: GitHub.