apache/seatunnel · error · CatalogException

Failed to create BigQuery dataset:

Error message

Failed to create BigQuery dataset: 

What it means

This CatalogException is thrown by BigQueryCatalog.createDatabase when the Google BigQuery client's bigquery.create(DatasetInfo...) call fails for any reason. It wraps the underlying exception (auth failure, quota, permission, name conflict, network) while naming the dataset that could not be created. It is the SeaTunnel catalog-layer translation of a BigQuery API failure during database (dataset) creation.

Source

Thrown at seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java:333

                    "Failed to drop BigQuery table: " + tablePath.getFullName(), e);
        }
    }

    @Override
    public void createDatabase(TablePath tablePath, boolean ignoreIfExists)
            throws DatabaseAlreadyExistException, CatalogException {
        String databaseName = getDatasetName(tablePath);
        if (databaseExists(databaseName)) {
            if (ignoreIfExists) {
                return;
            }
            throw new DatabaseAlreadyExistException(catalogName, databaseName);
        }
        try {
            bigquery.create(DatasetInfo.newBuilder(databaseName).build());
            log.info("BigQuery Dataset (database) '{}' created successfully.", databaseName);
        } catch (Exception e) {
            throw new CatalogException("Failed to create BigQuery dataset: " + databaseName, e);
        }
    }

    @Override
    public void dropDatabase(TablePath tablePath, boolean ignoreIfNotExists)
            throws DatabaseNotExistException, CatalogException {
        String databaseName = getDatasetName(tablePath);
        if (!databaseExists(databaseName)) {
            if (ignoreIfNotExists) {
                return;
            }
            throw new DatabaseNotExistException(catalogName, databaseName);
        }
        try {
            bigquery.delete(databaseName, BigQuery.DatasetDeleteOption.deleteContents());
            log.info("BigQuery Dataset (database) '{}' dropped successfully.", databaseName);
        } catch (Exception e) {
            throw new CatalogException("Failed to drop BigQuery dataset: " + databaseName, e);

View on GitHub (pinned to cf67b549a7)

Solutions

1. Check the wrapped cause 'e' in the stack trace for the real BigQuery error (409 conflict, 403 permission, 401 auth)
2. Verify the service account has roles/bigquery.dataOwner or bigquery.admin on the target project
3. Confirm the dataset does not already exist (or call catalog.databaseExists first) and the project ID in catalog options is correct
4. Validate GOOGLE_APPLICATION_CREDENTIALS points to a valid key file and retry transient network/quota failures

Example fix

// before
catalog.createDatabase(tablePath, false);
// after
if (!catalog.databaseExists(tablePath)) {
    catalog.createDatabase(tablePath, false);
}

if (catalog.databaseExists(tablePath)) {
    LOG.info("Dataset " + tablePath.getDatabaseName() + " already exists, skipping create");
} else {
    catalog.createDatabase(tablePath, false);
}

try {
    catalog.createDatabase(tablePath, false);
} catch (CatalogException e) {
    LOG.error("Dataset create failed; cause: " + e.getCause(), e);
    throw e;
}

When it happens

Trigger: Calling createDatabase(tablePath, ignoreIfNotExists) on a BigQueryCatalog when the service account lacks bigquery.datasets.create permission, the dataset already exists and the API rejects it, the project ID is wrong, credentials are missing/invalid, or the BigQuery API call hits a quota or network error.

Common situations: Wrong GOOGLE_APPLICATION_CREDENTIALS or a service account without BigQuery Admin role; dataset name colliding with an existing one (createDatabase does not check existence beforehand in the failing branch); misconfigured project; regional dataset restrictions; transient 429/5xx from the BigQuery API.

Related errors

• Failed to drop BigQuery dataset:
• Failed to open BigQueryCatalog
• Failed to list BigQuery databases (datasets)
• Failed to list tables in dataset:
• Failed to create BigQuery table:

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/37405316a54e6c9c. Report an issue: GitHub.