{"record":{"id":"b9652c97e95ae2bf","repo":"apache/beam","slug":"pipeline-options-cannot-be-null","errorCode":null,"errorMessage":"pipeline options cannot be null","messagePattern":"pipeline options cannot be null","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/DynamicDestinationsHelpers.java","lineNumber":446,"sourceCode":"        DynamicDestinations<T, DestinationT> inner, BigQueryServices bqServices) {\n      super(inner);\n      this.bqServices = bqServices;\n    }\n\n    private @Nullable Table getBigQueryTable(TableReference tableReference) {\n      BackOff backoff =\n          BackOffAdapter.toGcpBackOff(\n              FluentBackoff.DEFAULT\n                  .withMaxRetries(3)\n                  .withInitialBackoff(Duration.standardSeconds(1))\n                  .withMaxBackoff(Duration.standardSeconds(2))\n                  .backoff());\n      try {\n        do {\n          try {\n            PipelineOptions options = getPipelineOptions();\n            if (options == null) {\n              throw new IllegalStateException(\"pipeline options cannot be null\");\n            }\n            BigQueryOptions bqOptions = options.as(BigQueryOptions.class);\n            if (tableReference.getProjectId() == null) {\n              tableReference.setProjectId(\n                  bqOptions.getBigQueryProject() == null\n                      ? bqOptions.getProject()\n                      : bqOptions.getBigQueryProject());\n            }\n            try (DatasetService datasetService = bqServices.getDatasetService(bqOptions)) {\n              return datasetService.getTable(tableReference);\n            } catch (InterruptedException | IOException e) {\n              LOG.info(\"Failed to get BigQuery table {}\", tableReference);\n            }\n          } catch (Exception e) {\n            throw new RuntimeException(e);\n          }\n        } while (nextBackOff(Sleeper.DEFAULT, backoff));\n      } catch (InterruptedException e) {","sourceCodeStart":428,"sourceCodeEnd":464,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/DynamicDestinationsHelpers.java#L428-L464","documentation":"In DynamicDestinationsHelpers (table-creation helper), when resolving a BigQuery table the code fetches PipelineOptions via getPipelineOptions(); if null, IllegalStateException \"pipeline options cannot be null\" is thrown because project/projectId resolution requires BigQueryOptions.","triggerScenarios":"Calling existingTable/getBigQueryTable when the DynamicDestinations helper has no PipelineOptions set — typically when getPipelineOptions() was never wired from a DoFn.ProcessContext (options are set at pipeline setup), or invoking the helper outside a running pipeline (unit tests, direct construction).","commonSituations":"Unit-testing DynamicDestinations destinations without setting pipeline options; calling table-lookup helpers during pipeline construction before options are distributed; forgetting options.setPipelineOptions in the DoFn setup; serialized helper losing the options field.","solutions":["Set PipelineOptions on the object before the call (the helper's setPipelineOptions/setup path) — ensure BigQueryIO.Write pipeline setup runs it from the ProcessContext","In tests, call the options setter with TestPipeline.testingPipelineOptions() or PipelineOptionsFactory.create() before invoking existingTable","Ensure project ID is set explicitly on the TableReference so options-based defaulting isn't needed","Check that the DoFn's setup/startBundle wiring (setPipelineOptionsFromProcessContext) is not skipped"],"exampleFix":"// before: invoking helper in a test without options\nBigQueryServices.StorageClient ... helper = new TableSchemaUpdateTransaction(...);\nhelper.existingTable(ref);\n// after\nPipelineOptions options = PipelineOptionsFactory.create().as(BigQueryOptions.class);\noptions.setProject(\"my-project\");\nhelper.setPipelineOptions(options);\nhelper.existingTable(ref);","handlingStrategy":"validation","validationCode":"if (helper.getPipelineOptions() == null) {\n  helper.setPipelineOptions(PipelineOptionsFactory.create().as(BigQueryOptions.class));\n}","typeGuard":"boolean hasPipelineOptions(DynamicDestinations<?, ?> dd) { return dd.getPipelineOptions() != null; }","tryCatchPattern":"try { table = helper.existingTable(ref); } catch (IllegalStateException e) { LOG.error(\"Pipeline options missing: %s\", e.getMessage()); throw e; }","preventionTips":["Ensure BigQueryIO.Write's setup path wires pipeline options from the ProcessContext","In tests, always inject TestPipeline.testingPipelineOptions() before calling table helpers","Set the project ID explicitly on TableReferences to reduce options dependence","Check that serialized helper objects retain (or re-set) options after deserialization"],"tags":["bigquery","pipeline-options","illegal-state","null"],"backgroundTag":"null-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}