{"record":{"id":"c6aa4a02a36a613f","repo":"apache/beam","slug":"query-job-s-failed-status-s","errorCode":null,"errorMessage":"Query job %s failed, status: %s","messagePattern":"Query job (.+?) failed, status: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQueryHelper.java","lineNumber":200,"sourceCode":"\n      JobConfigurationQuery queryConfiguration =\n          createBasicQueryConfig(query, flattenResults, useLegacySql)\n              .setAllowLargeResults(true)\n              .setDestinationTable(queryResultTable)\n              .setCreateDisposition(\"CREATE_IF_NEEDED\")\n              .setWriteDisposition(\"WRITE_TRUNCATE\")\n              .setPriority(priority.name());\n\n      if (kmsKey != null) {\n        queryConfiguration.setDestinationEncryptionConfiguration(\n            new EncryptionConfiguration().setKmsKeyName(kmsKey));\n      }\n\n      JobService jobService = bqServices.getJobService(options);\n      jobService.startQueryJob(jobReference, queryConfiguration);\n      Job job = jobService.pollJob(jobReference, JOB_POLL_MAX_RETRIES);\n      if (BigQueryHelpers.parseStatus(job) != Status.SUCCEEDED) {\n        throw new IOException(\n            String.format(\n                \"Query job %s failed, status: %s\",\n                queryJobId, BigQueryHelpers.statusToPrettyString(job.getStatus())));\n      }\n\n      LOG.info(\"Query job {} completed\", queryJobId);\n      return queryResultTable;\n\n    } catch (RuntimeException | IOException | InterruptedException e) {\n      throw e;\n    } catch (Exception e) {\n      throw new RuntimeException(e);\n    }\n  }\n\n  private static JobConfigurationQuery createBasicQueryConfig(\n      String query, Boolean flattenResults, Boolean useLegacySql) {\n    return new JobConfigurationQuery()","sourceCodeStart":182,"sourceCodeEnd":218,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryQueryHelper.java#L182-L218","documentation":"BigQueryIo throws this IOException in BigQueryQueryHelper.executeQuery when a submitted BigQuery query job does not reach SUCCEEDED status after polling. The job was successfully started via JobService.startQueryJob, but BigQueryHelpers.parseStatus reported a non-successful terminal state (FAILED, UNKNOWN, etc.). The message includes the query job id and a pretty-printed job status with error details.","triggerScenarios":"Executing a BigQueryIO query (e.g. BigQueryIO.readTableRows().fromQuery(...)) where the query is invalid (syntax error, unknown table/column), the job exceeds quotas, or the service returns a FAILED status during jobService.pollJob within JOB_POLL_MAX_RETRIES.","commonSituations":"Typo in SQL or dataset/table names; referencing tables the caller lacks permission to read; query exceeding the per-project concurrent quota or slot limits; using legacy SQL dialect while useLegacySql is false (or vice versa); job killed or failed server-side before completion.","solutions":["Inspect the pretty-printed status in the message (and job.getStatus().getErrors()) for the underlying BigQuery error reason","Run the query text manually in the BigQuery console to reproduce and fix SQL syntax or reference errors","Verify the referenced datasets/tables exist and the service account has bigquery.jobs.create and read permissions","Check project quota/usage if the status indicates rate or resource limits","Confirm the SQL dialect matches the useLegacySql configuration"],"exampleFix":"// before: silently failing query with wrong dialect\nBigQueryIO.readTableRows().fromQuery(\"SELECT * FROM my_dataset.my_table\")\n// after: validate the query runs before the pipeline, and set dialect explicitly\nBQ QueryUtils.validateQuery(\"SELECT * FROM `my-project.my_dataset.my_table`\", /*useLegacySql=*/ false);\nBigQueryIO.readTableRows().fromQuery(\"SELECT * FROM `my-project.my_dataset.my_table`\").withMethod(Method.QUERY)","handlingStrategy":"try-catch","validationCode":"// Validate the query before running the pipeline (dry run)\nQueryRequest dryRun = QueryRequest.newBuilder(query).setUseLegacySql(false).setDryRun(true).build();\ndataset.query(dryRun); // throws early with the real BigQuery validation error","typeGuard":"// Java: check job status before relying on results\nboolean jobSucceeded(com.google.api.services.bigquery.model.Job job) {\n  return job != null && job.getStatus() != null && \"DONE\".equals(job.getStatus().getState())\n      && job.getStatus().getErrorResult() == null;\n}","tryCatchPattern":"try {\n  pipeline.apply(BigQueryIO.readTableRows().fromQuery(query));\n} catch (IOException e) {\n  if (e.getMessage().contains(\"Query job\") && e.getMessage().contains(\"failed\")) {\n    LOG.error(\"BigQuery query job failed; check SQL, permissions and quotas\", e);\n    throw new ValidationException(\"Invalid BigQuery query: \" + e.getCause(), e);\n  }\n  throw e;\n}","preventionTips":["Dry-run every query in CI before deploying the pipeline","Use fully qualified standard-SQL table names (project.dataset.table) with useLegacySql=false","Verify service account permissions (bigquery.jobs.create, table read) per project","Monitor BigQuery quotas and slot usage for query-heavy pipelines"],"tags":["bigquery","gcp","query-job","io-exception"],"backgroundTag":"database-query-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}