{"record":{"id":"869b0529e7c3ed3d","repo":"apache/beam","slug":"unable-to-confirm-bigquery-table-emptiness-for-table-s","errorCode":null,"errorMessage":"unable to confirm BigQuery table emptiness for table %s","messagePattern":"unable to confirm BigQuery table emptiness for table (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java","lineNumber":676,"sourceCode":"  static String randomUUIDString() {\n    return UUID.randomUUID().toString().replaceAll(\"-\", \"\");\n  }\n\n  static void verifyTableNotExistOrEmpty(DatasetService datasetService, TableReference tableRef) {\n    try {\n      if (datasetService.getTable(\n              tableRef, Collections.emptyList(), DatasetService.TableMetadataView.BASIC)\n          != null) {\n        checkState(\n            datasetService.isTableEmpty(tableRef),\n            \"BigQuery table is not empty: %s.\",\n            toTableSpec(tableRef));\n      }\n    } catch (IOException | InterruptedException e) {\n      if (e instanceof InterruptedException) {\n        Thread.currentThread().interrupt();\n      }\n      throw new RuntimeException(\n          \"unable to confirm BigQuery table emptiness for table \" + toTableSpec(tableRef), e);\n    }\n  }\n\n  static void verifyDatasetPresence(DatasetService datasetService, TableReference table) {\n    try {\n      datasetService.getDataset(table.getProjectId(), table.getDatasetId());\n    } catch (Exception e) {\n      ApiErrorExtractor errorExtractor = new ApiErrorExtractor();\n      if ((e instanceof IOException) && errorExtractor.itemNotFound((IOException) e)) {\n        throw new IllegalArgumentException(\n            String.format(RESOURCE_NOT_FOUND_ERROR, \"dataset\", toTableSpec(table)), e);\n      } else if (e instanceof RuntimeException) {\n        throw (RuntimeException) e;\n      } else {\n        throw new RuntimeException(\n            String.format(\n                UNABLE_TO_CONFIRM_PRESENCE_OF_RESOURCE_ERROR, \"dataset\", toTableSpec(table)),","sourceCodeStart":658,"sourceCodeEnd":694,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java#L658-L694","documentation":"During pipeline validation, Beam queries the BigQuery API to confirm whether a target table exists and is empty. If that verification call fails with IOException or InterruptedException, it cannot determine table emptiness and wraps the error in a RuntimeException mentioning the table spec. The original exception is preserved as the cause.","triggerScenarios":"BigQueryIO write with validation enabled calling verifyTableEmptyness when the tabledata/jobs API call throws IOException (network, quota, permissions) or the thread is interrupted while checking the table.","commonSituations":"Transient GCP network issues or token refresh failures during pipeline startup, insufficient IAM permissions to read the table, quota/rate limiting, or Dataflow worker shutdown interrupting the check.","solutions":["Inspect the cause to see whether it's a network error, quota issue, or permission problem and fix that root cause.","Grant the pipeline service account BigQuery Data Viewer/Job User roles.","Disable validation with .withoutValidation() if you intentionally don't need pre-write emptiness checks."],"exampleFix":"// before\nBigQueryIO.writeTableRows().to(spec).withCreateDisposition(CREATE_IF_NEEDED); // validation on\n\n// after\nBigQueryIO.writeTableRows().to(spec)\n    .withCreateDisposition(CREATE_IF_NEEDED)\n    .withoutValidation();","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  pipeline.run().waitUntilFinish();\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"unable to confirm BigQuery table emptiness\")) {\n    // inspect cause (IOException/InterruptedException) and retry after network/quota check\n  }\n}","preventionTips":["Ensure reliable network/auth from workers to bigquery.googleapis.com.","Grant read permissions on the target table's project to the service account.","Use .withoutValidation() when pre-flight checks aren't required.","Retry pipelines on transient API errors instead of failing permanently."],"tags":["java","bigquery","api-error","validation"],"backgroundTag":"api-request-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}