apache/beam · error · IllegalArgumentException

Validation of query "%1$s" failed. If the query depends on a

Error message

Validation of query "%1$s" failed. If the query depends on an earlier stage of the pipeline, This validation can be disabled using #withoutValidation.

What it means

Thrown when dry-run validation of the SQL query fails at pipeline-construction/validation time. Beam executes a query dry run against BigQuery; any error from that call is wrapped in this IllegalArgumentException.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java:1491

                query.isAccessible(), "Cannot call validate if query is dynamically set.");
            JobService jobService = getBigQueryServices().getJobService(bqOptions);
            // JobConfigurationQuery accepts null for both of these, but the generated API client
            // is not annotated.
            @SuppressWarnings("nullness")
            JobConfigurationQuery queryConfig =
                new JobConfigurationQuery()
                    .setQuery(query.get())
                    .setFlattenResults(getFlattenResults())
                    .setUseLegacySql(getUseLegacySql());
            try {
              jobService.dryRunQuery(
                  bqOptions.getBigQueryProject() == null
                      ? bqOptions.getProject()
                      : bqOptions.getBigQueryProject(),
                  queryConfig,
                  getQueryLocation());
            } catch (Exception e) {
              throw new IllegalArgumentException(
                  String.format(
                      "Validation of query \"%1$s\" failed. If the query depends on an earlier stage of the"
                          + " pipeline, This validation can be disabled using #withoutValidation.",
                      query.get()),
                  e);
            }

            // If the user provided a temp dataset, check if the dataset exists before launching the
            // query
            String queryTempDataset = getQueryTempDataset();
            if (queryTempDataset != null) {
              // The temp table is only used for dataset and project id validation, not for table
              // name
              // validation
              String project = getQueryTempProject();
              if (project == null) {
                project =
                    bqOptions.getBigQueryProject() == null

View on GitHub (pinned to 12126d8942)

Solutions

1. If the query depends on earlier pipeline output, disable validation with .withoutValidation()
2. Fix the SQL error reported in the chained exception (run the query in the BigQuery console)
3. Check BigQuery permissions and query location via .withQueryLocation()

Example fix

// before
BigQueryIO.readTableRows().fromQuery("SELECT * FROM proj:ds.tbl");
// after
BigQueryIO.readTableRows().fromQuery("SELECT * FROM proj:ds.tbl").withoutValidation();

try {
  p.run().waitUntilFinish();
} catch (IllegalArgumentException e) {
  if (e.getMessage() != null && e.getMessage().contains("Validation of query")) {
    // dry run failed: check SQL, permissions, or use withoutValidation()
  }
  throw e;
}

When it happens

Trigger: BigQueryIO.readQuery/readTableRows(query) with validation enabled (default) and the dry run fails: invalid SQL, unknown table/column, or insufficient permissions.

Common situations: Query references tables created by an earlier pipeline stage; SQL syntax errors; missing BigQuery Job dry-run permissions; wrong query location/project.

Understand the failure class

Background: "query failed", "%w: SQL error" — wrapped database query errors in Go libraries explained — this error's family across 3 libraries.

Related errors

• unable to confirm BigQuery table emptiness for table %s
• BigQuery %1$s not found for table "%2$s" . Please create the
• Unable to confirm BigQuery %1$s presence for table "%2$s". I
• invalid table ID %s. Table IDs must be alphanumeric (plus un
• Primary key validation error! Multiple inserts with the same

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9058fb2db35fdc46. Report an issue: GitHub.