{"record":{"id":"f296d6b962d38fd0","repo":"apache/beam","slug":"unable-to-obtain-dataset-for-dataset-s-in-project-s","errorCode":null,"errorMessage":"unable to obtain dataset for dataset %s in project %s","messagePattern":"unable to obtain dataset for dataset (.+?) in project (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java","lineNumber":741,"sourceCode":"    try (DatasetService datasetService = new BigQueryServicesImpl().getDatasetService(options)) {\n      return datasetService.getTable(tableRef);\n    } catch (IOException | InterruptedException e) {\n      throw e;\n    } catch (Exception e) {\n      throw new RuntimeException(e);\n    }\n  }\n\n  static String getDatasetLocation(\n      DatasetService datasetService, String projectId, String datasetId) {\n    Dataset dataset;\n    try {\n      dataset = datasetService.getDataset(projectId, datasetId);\n    } catch (Exception e) {\n      if (e instanceof InterruptedException) {\n        Thread.currentThread().interrupt();\n      }\n      throw new RuntimeException(\n          String.format(\n              \"unable to obtain dataset for dataset %s in project %s\", datasetId, projectId),\n          e);\n    }\n    return dataset.getLocation();\n  }\n\n  static void verifyTablePresence(DatasetService datasetService, TableReference table) {\n    try {\n      Table fetchedTable = datasetService.getTable(table);\n      if (fetchedTable == null) {\n        throw new IOException(\"Table does not exist.\");\n      }\n    } catch (Exception e) {\n      ApiErrorExtractor errorExtractor = new ApiErrorExtractor();\n      if ((e instanceof IOException)\n          && (\"Table does not exist.\".equals(e.getMessage())\n              || errorExtractor.itemNotFound((IOException) e))) {","sourceCodeStart":723,"sourceCodeEnd":759,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java#L723-L759","documentation":"While resolving a dataset's location (needed to pick the right BigQuery service endpoint), BigQueryHelpers calls datasetService.getDataset. If that call throws any Exception, it re-wraps it in a RuntimeException 'unable to obtain dataset for dataset %s in project %s' with the original as cause, and interrupts restore if the thread was interrupted.","triggerScenarios":"Calling the helper that resolves dataset location (used by BigQueryIO to select regional endpoints) when datasets.get fails: network errors, quota, IAM denial, or thread interruption.","commonSituations":"Dataset name/project typo (dataset genuinely absent), service account without bigquery.datasets.get, firewall blocking googleapis.com from workers, or Dataflow cancel/drain interrupting the call.","solutions":["Check the cause for the real error; verify the dataset exists: bq show my-project:my_dataset.","Grant the service account bigquery.datasets.get (BigQuery Metadata Viewer) and ensure network access to bigquery.googleapis.com.","If the thread was interrupted, let shutdown proceed and re-run; avoid swallowing InterruptedException."],"exampleFix":"// before\n// RuntimeException: unable to obtain dataset for dataset analytics in project my-project\n\n// after\n// bq mk --dataset my-project:analytics  (or fix the project/dataset name), then re-run the pipeline","handlingStrategy":"try-catch","validationCode":"Dataset ds = bigquery.getDataset(projectId, datasetId);\nif (ds == null) {\n  throw new IllegalStateException(\"dataset \" + datasetId + \" missing in project \" + projectId);\n}","typeGuard":null,"tryCatchPattern":"try {\n  String location = helper.resolveDatasetLocation(projectId, datasetId);\n} catch (RuntimeException e) {\n  // inspect cause; check dataset existence, IAM, and network before retry\n}","preventionTips":["Verify the dataset exists in the expected project before pipeline launch.","Grant metadata-read IAM permissions to the pipeline service account.","Ensure egress to bigquery.googleapis.com from worker networks.","Never swallow InterruptedException; allow clean shutdown on cancel/drain."],"tags":["java","bigquery","api-error","dataset-location"],"backgroundTag":"api-request-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}