{"record":{"id":"087f81e5155f20a0","repo":"apache/beam","slug":"unable-to-confirm-bigquery-1-s-presence-for-table-2-s-if-the","errorCode":null,"errorMessage":"Unable to confirm BigQuery %1$s presence for table \"%2$s\". If the %1$s is created by an earlier stage of the pipeline, this validation can be disabled using #withoutValidation.","messagePattern":"Unable to confirm BigQuery %1\\$s presence for table \"%2\\$s\"\\. If the %1\\$s is created by an earlier stage of the pipeline, this validation can be disabled using #withoutValidation\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java","lineNumber":692,"sourceCode":"        Thread.currentThread().interrupt();\n      }\n      throw new RuntimeException(\n          \"unable to confirm BigQuery table emptiness for table \" + toTableSpec(tableRef), e);\n    }\n  }\n\n  static void verifyDatasetPresence(DatasetService datasetService, TableReference table) {\n    try {\n      datasetService.getDataset(table.getProjectId(), table.getDatasetId());\n    } catch (Exception e) {\n      ApiErrorExtractor errorExtractor = new ApiErrorExtractor();\n      if ((e instanceof IOException) && errorExtractor.itemNotFound((IOException) e)) {\n        throw new IllegalArgumentException(\n            String.format(RESOURCE_NOT_FOUND_ERROR, \"dataset\", toTableSpec(table)), e);\n      } else if (e instanceof RuntimeException) {\n        throw (RuntimeException) e;\n      } else {\n        throw new RuntimeException(\n            String.format(\n                UNABLE_TO_CONFIRM_PRESENCE_OF_RESOURCE_ERROR, \"dataset\", toTableSpec(table)),\n            e);\n      }\n    }\n  }\n\n  /**\n   * It returns the number of rows for a given table.\n   *\n   * @return The number of rows in the table or null if it cannot get any estimate.\n   */\n  public static @Nullable BigInteger getNumRows(BigQueryOptions options, TableReference tableRef)\n      throws InterruptedException, IOException {\n\n    try (DatasetService datasetService = new BigQueryServicesImpl().getDatasetService(options)) {\n      Table table = datasetService.getTable(tableRef);\n      if (table == null) {","sourceCodeStart":674,"sourceCodeEnd":710,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java#L674-L710","documentation":"Also from verifyDatasetPresence(): when the datasets.get call fails but the error is neither a recognizable itemNotFound IOException nor a RuntimeException, the library can't classify it and throws a generic RuntimeException (UNABLE_TO_CONFIRM_PRESENCE_OF_RESOURCE_ERROR) preserving the cause. It indicates validation could not be completed, not that the dataset is definitely missing.","triggerScenarios":"datasets.get throws a non-404, non-RuntimeException error during validation: network failures, 401/403 auth errors surfaced as checked exceptions, quota errors, or service unavailability.","commonSituations":"Expired/invalid credentials, IAM permission lacking bigquery.datasets.get, VPC/SCC network restrictions from the worker, or transient BigQuery outages during pipeline launch.","solutions":["Inspect the cause exception for the concrete HTTP status and fix the underlying auth/network/quota issue.","Grant the service account BigQuery Metadata Viewer (bigquery.datasets.get) permission.","Use .withoutValidation() to skip pre-flight checks if your environment intentionally restricts metadata reads."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  pipeline.run().waitUntilFinish();\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Unable to confirm BigQuery\")) {\n    // inspect cause: auth (401/403), network, quota; fix then re-run\n  }\n}","preventionTips":["Grant bigquery.datasets.get (BigQuery Metadata Viewer) to the service account.","Confirm workers can reach googleapis.com (VPC/SCP/firewall rules).","Refresh credentials and check token validity before long runs.","Disable validation with #withoutValidation if metadata reads are restricted."],"tags":["java","bigquery","api-error","validation","permissions"],"backgroundTag":"api-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}