apache/seatunnel · error · CatalogException

Failed to list tables in dataset:

Error message

Failed to list tables in dataset: 

What it means

BigQueryCatalog.listTables(databaseName) wraps any exception from bigquery.listTables(databaseName) / iterateAll() in a CatalogException reporting which dataset failed. BigQuery 'datasets' are SeaTunnel 'databases', so this is the table-listing RPC failing for the given dataset. The real cause (permissions, nonexistent dataset, network) is attached as the exception cause.

Source

Thrown at seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java:184

    }

    @Override
    public List<String> listTables(String databaseName)
            throws CatalogException, DatabaseNotExistException {
        if (databaseName == null || databaseName.trim().isEmpty()) {
            databaseName = config.get(BigQuerySinkOptions.DATASET_ID);
        }
        if (!databaseExists(databaseName)) {
            throw new DatabaseNotExistException(catalogName, databaseName);
        }
        List<String> tables = new ArrayList<>();
        try {
            Page<Table> bqTables = bigquery.listTables(databaseName);
            for (Table table : bqTables.iterateAll()) {
                tables.add(table.getTableId().getTable());
            }
        } catch (Exception e) {
            throw new CatalogException("Failed to list tables in dataset: " + databaseName, e);
        }
        return tables;
    }

    @Override
    public boolean tableExists(TablePath tablePath) throws CatalogException {
        TableId tableId = TableId.of(getDatasetName(tablePath), tablePath.getTableName());
        return bigquery.getTable(tableId) != null;
    }

    @Override
    public CatalogTable getTable(TablePath tablePath)
            throws CatalogException, TableNotExistException {
        TableId tableId = TableId.of(getDatasetName(tablePath), tablePath.getTableName());
        Table table = bigquery.getTable(tableId);
        if (table == null) {
            throw new TableNotExistException(catalogName, tablePath);
        }

View on GitHub (pinned to cf67b549a7)

Solutions

1. Verify the dataset exists: `bq ls --project_id=PROJECT` and confirm `databaseName` matches exactly (dataset names are case-sensitive).
2. Confirm the service account has `roles/bigquery.dataViewer` on that dataset (or project) so it can call tables.list.
3. Check the configured project id matches the project that hosts the dataset; use the wrapped cause to see if BigQuery returned 404 'not found' vs 403.
4. Re-validate credentials as for listDatabases (GOOGLE_APPLICATION_CREDENTIALS, `bq ls` smoke test).
5. For very large datasets or rate-limit causes, retry with backoff or narrow the scope of the job.

Example fix

// before: job config referencing a dataset that doesn't exist
// catalog_table_path = "bigquery.wrong_dataset.my_table"

// after: use the actual dataset name
// catalog_table_path = "bigquery.analytics.my_table"

// Java, before calling catalog.listTables(dataset)
// check dataset exists via the BigQuery client used by the catalog
DatasetId dsId = DatasetId.of(projectId, datasetName);
if (bigquery.getDataset(dsId) == null) {
    throw new IllegalArgumentException(
        "Dataset does not exist: " + project + "." + datasetName);
}
// smoke-test table listing permission
bigquery.listTables(dsId, BigQuery.TableListOption.pageSize(1));

static boolean datasetExists(BigQuery bq, String project, String dataset) {
    try {
        return bq.getDataset(DatasetId.of(project, dataset)) != null;
    } catch (BigQueryException e) {
        return false;
    }
}

try {
    tables = catalog.listTables(datasetName);
} catch (CatalogException e) {
    if (e.getCause() instanceof BigQueryException be && be.getCode() == 404) {
        // dataset name/project typo — fail fast with a clear message
        throw new IllegalArgumentException("Unknown dataset: " + datasetName, e);
    }
    throw e;
}

When it happens

Trigger: Any exception thrown by the BigQuery client while listing tables in `databaseName`: dataset does not exist in the project, caller lacks bigquery.tables.list permission on the dataset, project id mismatch, credentials invalid, network/timeout to bigquery.googleapis.com, or rate limiting while paginating with iterateAll().

Common situations: Typo in the dataset name in the SeaTunnel table path (database = dataset); dataset exists in a different project than the one configured; a recently deleted dataset still referenced by a job; service account scoped to another project; transient 429s on projects with thousands of tables.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Failed to list BigQuery databases (datasets)
• Failed to create BigQuery table:
• Failed to drop BigQuery table:
• Failed to open BigQueryCatalog
• Failed to create BigQuery dataset:

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/16e656d60b308697. Report an issue: GitHub.