{"record":{"id":"16e656d60b308697","repo":"apache/seatunnel","slug":"failed-to-list-tables-in-dataset","errorCode":null,"errorMessage":"Failed to list tables in dataset: ","messagePattern":"Failed to list tables in dataset: ","errorType":"exception","errorClass":"CatalogException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java","lineNumber":184,"sourceCode":"    }\n\n    @Override\n    public List<String> listTables(String databaseName)\n            throws CatalogException, DatabaseNotExistException {\n        if (databaseName == null || databaseName.trim().isEmpty()) {\n            databaseName = config.get(BigQuerySinkOptions.DATASET_ID);\n        }\n        if (!databaseExists(databaseName)) {\n            throw new DatabaseNotExistException(catalogName, databaseName);\n        }\n        List<String> tables = new ArrayList<>();\n        try {\n            Page<Table> bqTables = bigquery.listTables(databaseName);\n            for (Table table : bqTables.iterateAll()) {\n                tables.add(table.getTableId().getTable());\n            }\n        } catch (Exception e) {\n            throw new CatalogException(\"Failed to list tables in dataset: \" + databaseName, e);\n        }\n        return tables;\n    }\n\n    @Override\n    public boolean tableExists(TablePath tablePath) throws CatalogException {\n        TableId tableId = TableId.of(getDatasetName(tablePath), tablePath.getTableName());\n        return bigquery.getTable(tableId) != null;\n    }\n\n    @Override\n    public CatalogTable getTable(TablePath tablePath)\n            throws CatalogException, TableNotExistException {\n        TableId tableId = TableId.of(getDatasetName(tablePath), tablePath.getTableName());\n        Table table = bigquery.getTable(tableId);\n        if (table == null) {\n            throw new TableNotExistException(catalogName, tablePath);\n        }","sourceCodeStart":166,"sourceCodeEnd":202,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-bigquery/src/main/java/org/apache/seatunnel/connectors/bigquery/catalog/BigQueryCatalog.java#L166-L202","documentation":"BigQueryCatalog.listTables(databaseName) wraps any exception from bigquery.listTables(databaseName) / iterateAll() in a CatalogException reporting which dataset failed. BigQuery 'datasets' are SeaTunnel 'databases', so this is the table-listing RPC failing for the given dataset. The real cause (permissions, nonexistent dataset, network) is attached as the exception cause.","triggerScenarios":"Any exception thrown by the BigQuery client while listing tables in `databaseName`: dataset does not exist in the project, caller lacks bigquery.tables.list permission on the dataset, project id mismatch, credentials invalid, network/timeout to bigquery.googleapis.com, or rate limiting while paginating with iterateAll().","commonSituations":"Typo in the dataset name in the SeaTunnel table path (database = dataset); dataset exists in a different project than the one configured; a recently deleted dataset still referenced by a job; service account scoped to another project; transient 429s on projects with thousands of tables.","solutions":["Verify the dataset exists: `bq ls --project_id=PROJECT` and confirm `databaseName` matches exactly (dataset names are case-sensitive).","Confirm the service account has `roles/bigquery.dataViewer` on that dataset (or project) so it can call tables.list.","Check the configured project id matches the project that hosts the dataset; use the wrapped cause to see if BigQuery returned 404 'not found' vs 403.","Re-validate credentials as for listDatabases (GOOGLE_APPLICATION_CREDENTIALS, `bq ls` smoke test).","For very large datasets or rate-limit causes, retry with backoff or narrow the scope of the job."],"exampleFix":"// before: job config referencing a dataset that doesn't exist\n// catalog_table_path = \"bigquery.wrong_dataset.my_table\"\n\n// after: use the actual dataset name\n// catalog_table_path = \"bigquery.analytics.my_table\"","handlingStrategy":"validation","validationCode":"// Java, before calling catalog.listTables(dataset)\n// check dataset exists via the BigQuery client used by the catalog\nDatasetId dsId = DatasetId.of(projectId, datasetName);\nif (bigquery.getDataset(dsId) == null) {\n    throw new IllegalArgumentException(\n        \"Dataset does not exist: \" + project + \".\" + datasetName);\n}\n// smoke-test table listing permission\nbigquery.listTables(dsId, BigQuery.TableListOption.pageSize(1));","typeGuard":"static boolean datasetExists(BigQuery bq, String project, String dataset) {\n    try {\n        return bq.getDataset(DatasetId.of(project, dataset)) != null;\n    } catch (BigQueryException e) {\n        return false;\n    }\n}","tryCatchPattern":"try {\n    tables = catalog.listTables(datasetName);\n} catch (CatalogException e) {\n    if (e.getCause() instanceof BigQueryException be && be.getCode() == 404) {\n        // dataset name/project typo — fail fast with a clear message\n        throw new IllegalArgumentException(\"Unknown dataset: \" + datasetName, e);\n    }\n    throw e;\n}","preventionTips":["Match dataset names exactly (case-sensitive) against `bq ls` output","Ensure the configured project id equals the dataset's project","Grant dataViewer on the dataset for tables.list access","Validate table paths at config-load time, not mid-job","Watch for 429 on huge datasets and add backoff"],"tags":["bigquery","gcp","metadata","permissions","dataset"],"backgroundTag":"resource-not-found","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}