{"record":{"id":"98e40140c03f1591","repo":"apache/beam","slug":"bigquery-1-s-not-found-for-table-2-s-please-create-the-1-s","errorCode":null,"errorMessage":"BigQuery %1$s not found for table \"%2$s\" . Please create the %1$s before pipeline execution. If the %1$s is created by an earlier stage of the pipeline, this validation can be disabled using #withoutValidation.","messagePattern":"BigQuery %1\\$s not found for table \"%2\\$s\" \\. Please create the %1\\$s before pipeline execution\\. If the %1\\$s is created by an earlier stage of the pipeline, this validation can be disabled using #withoutValidation\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":404,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java","lineNumber":687,"sourceCode":"            \"BigQuery table is not empty: %s.\",\n            toTableSpec(tableRef));\n      }\n    } catch (IOException | InterruptedException e) {\n      if (e instanceof InterruptedException) {\n        Thread.currentThread().interrupt();\n      }\n      throw new RuntimeException(\n          \"unable to confirm BigQuery table emptiness for table \" + toTableSpec(tableRef), e);\n    }\n  }\n\n  static void verifyDatasetPresence(DatasetService datasetService, TableReference table) {\n    try {\n      datasetService.getDataset(table.getProjectId(), table.getDatasetId());\n    } catch (Exception e) {\n      ApiErrorExtractor errorExtractor = new ApiErrorExtractor();\n      if ((e instanceof IOException) && errorExtractor.itemNotFound((IOException) e)) {\n        throw new IllegalArgumentException(\n            String.format(RESOURCE_NOT_FOUND_ERROR, \"dataset\", toTableSpec(table)), e);\n      } else if (e instanceof RuntimeException) {\n        throw (RuntimeException) e;\n      } else {\n        throw new RuntimeException(\n            String.format(\n                UNABLE_TO_CONFIRM_PRESENCE_OF_RESOURCE_ERROR, \"dataset\", toTableSpec(table)),\n            e);\n      }\n    }\n  }\n\n  /**\n   * It returns the number of rows for a given table.\n   *\n   * @return The number of rows in the table or null if it cannot get any estimate.\n   */\n  public static @Nullable BigInteger getNumRows(BigQueryOptions options, TableReference tableRef)","sourceCodeStart":669,"sourceCodeEnd":705,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java#L669-L705","documentation":"verifyDatasetPresence() checks that the dataset backing the target table exists before running the pipeline. When the BigQuery API returns an itemNotFound error for datasets.get, the library throws IllegalArgumentException with RESOURCE_NOT_FOUND_ERROR telling the developer to create the dataset first or disable validation with #withoutValidation.","triggerScenarios":"BigQueryIO write with validation enabled where the referenced dataset does not exist in the project at pipeline submission time (datasets.get returns 404).","commonSituations":"Typo in the dataset name, wrong project, dataset expected to be created by an earlier pipeline stage but validation runs before it, or environment (prod vs staging) mismatch.","solutions":["Create the dataset before running the pipeline (bq mk or Cloud Console).","Fix typos / verify the correct project and dataset in the table spec.","If the dataset is created by an earlier stage of the same pipeline, disable validation with .withoutValidation()."],"exampleFix":"// before\nString spec = \"my-project:nonexistent_dataset.my_table\"; // dataset missing\n\n// after\n// run: bq mk --dataset my-project:nonexistent_dataset\nString spec = \"my-project:nonexistent_dataset.my_table\";","handlingStrategy":"validation","validationCode":"// before running the pipeline\nDataset dataset = bigquery.getDataset(projectId, datasetId);\nif (dataset == null) {\n  throw new IllegalStateException(\"Dataset \" + datasetId + \" does not exist in \" + projectId);\n}","typeGuard":null,"tryCatchPattern":"try {\n  pipeline.run().waitUntilFinish();\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().contains(\"not found\") && e.getMessage().contains(\"dataset\")) {\n    // create the dataset or switch to withoutValidation()\n  }\n}","preventionTips":["Verify dataset existence with 'bq show project:dataset' before submission.","Create datasets via IaC/CLI before deploying pipelines that depend on them.","Avoid typos by building specs from constants/config rather than ad-hoc strings.","Use #withoutValidation when resources are created mid-pipeline."],"tags":["java","bigquery","resource-not-found","validation"],"backgroundTag":"resource-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}