{"record":{"id":"4f647a20fa93f486","repo":"apache/beam","slug":"missing-required-configuration-parameters-s","errorCode":null,"errorMessage":"Missing required configuration parameters: %s","messagePattern":"Missing required configuration parameters: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/dataflow/internal/apiclient.py","lineNumber":415,"sourceCode":"      job_name = Job._build_default_job_name(getpass.getuser())\n    return job_name\n\n  def __init__(self, options, proto_pipeline):\n    self.options = options\n    validate_pipeline_graph(proto_pipeline)\n    self.proto_pipeline = proto_pipeline\n    self.google_cloud_options = options.view_as(GoogleCloudOptions)\n    if not self.google_cloud_options.job_name:\n      self.google_cloud_options.job_name = self.default_job_name(\n          self.google_cloud_options.job_name)\n\n    required_google_cloud_options = ['project', 'job_name', 'temp_location']\n    missing = [\n        option for option in required_google_cloud_options\n        if not getattr(self.google_cloud_options, option)\n    ]\n    if missing:\n      raise ValueError(\n          'Missing required configuration parameters: %s' % missing)\n\n    if not self.google_cloud_options.staging_location:\n      _LOGGER.info(\n          'Defaulting to the temp_location as staging_location: %s',\n          self.google_cloud_options.temp_location)\n      (\n          self.google_cloud_options.staging_location\n      ) = self.google_cloud_options.temp_location\n\n    self.root_staging_location = self.google_cloud_options.staging_location\n\n    # Make the staging and temp locations job name and time specific. This is\n    # needed to avoid clashes between job submissions using the same staging\n    # area or team members using same job names. This method is not entirely\n    # foolproof since two job submissions with same name can happen at exactly\n    # the same time. However the window is extremely small given that\n    # time.time() has at least microseconds granularity. We add the suffix only","sourceCodeStart":397,"sourceCodeEnd":433,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py#L397-L433","documentation":"ValueError raised in DataflowPipelineOptions validation when any of the required Google Cloud options — project, job_name, or temp_location — is missing or falsy. Dataflow cannot identify the GCP project, name the job, or stage temporary files without these.","triggerScenarios":"Running with the DataflowRunner while PipelineOptions lacks --project, --job_name, or --temp_location (or they are set to empty strings).","commonSituations":"Local runs that worked with DirectRunner being submitted to Dataflow without adding GCP options; CI pipelines missing flags; job_name or project accidentally overridden to None by config loading.","solutions":["Pass all three options: --project, --job_name, and --temp_location (a GCS path like gs://bucket/temp)","Set them programmatically via GoogleCloudOptions before calling pipeline.run()","Verify the options object actually carries the values (e.g. print(options.view_as(GoogleCloudOptions).project)) — config-file keys may not map as expected"],"exampleFix":"# before\noptions = PipelineOptions(['--runner=DataflowRunner'])\n# after\noptions = PipelineOptions([\n    '--runner=DataflowRunner',\n    '--project=my-gcp-project',\n    '--job_name=my-job',\n    '--temp_location=gs://my-bucket/temp'])","handlingStrategy":"validation","validationCode":"gco = options.view_as(GoogleCloudOptions)\nmissing = [o for o in ('project', 'job_name', 'temp_location') if not getattr(gco, o)]\nif missing:\n    raise SystemExit('missing: %s' % missing)","typeGuard":null,"tryCatchPattern":"try:\n    pipeline.run()\nexcept ValueError as e:\n    if 'Missing required configuration' in str(e):\n        print('Add --project/--job_name/--temp_location')\n    raise","preventionTips":["Keep a shared options template including project, job_name, temp_location, region","Validate GoogleCloudOptions before pipeline.run() in CI","Avoid empty-string values — they are treated as missing"],"tags":["dataflow","configuration","gcp","cli-options"],"backgroundTag":"missing-required-config-field","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}