{"record":{"id":"4d972a0dc1056453","repo":"apache/beam","slug":"the-staging-location-option-must-be-specified","errorCode":null,"errorMessage":"The --staging_location option must be specified.","messagePattern":"The --staging_location option must be specified\\.","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/dataflow/internal/apiclient.py","lineNumber":602,"sourceCode":"          to_path,\n          cached_path)\n    else:\n      self._uncached_gcs_file_copy(from_path, cached_path)\n\n    FileSystems.copy(\n        source_file_names=[cached_path], destination_file_names=[to_path])\n    _LOGGER.info('Copied cached artifact from %s to %s', from_path, to_path)\n\n  def _uncached_gcs_file_copy(self, from_path, to_path):\n    to_folder, to_name = os.path.split(to_path)\n    total_size = os.path.getsize(from_path)\n    self.stage_file_with_retry(\n        to_folder, to_name, from_path, total_size=total_size)\n\n  def _stage_resources(self, pipeline, options):\n    google_cloud_options = options.view_as(GoogleCloudOptions)\n    if google_cloud_options.staging_location is None:\n      raise RuntimeError('The --staging_location option must be specified.')\n    if google_cloud_options.temp_location is None:\n      raise RuntimeError('The --temp_location option must be specified.')\n\n    resources = []\n    staged_paths = {}\n    staged_hashes = {}\n    for _, env in sorted(pipeline.components.environments.items(),\n                         key=lambda kv: kv[0]):\n      for dep in env.dependencies:\n        if dep.type_urn != common_urns.artifact_types.FILE.urn:\n          raise RuntimeError('unsupported artifact type %s' % dep.type_urn)\n        type_payload = beam_runner_api_pb2.ArtifactFilePayload.FromString(\n            dep.type_payload)\n\n        if dep.role_urn == common_urns.artifact_roles.STAGING_TO.urn:\n          remote_name = (\n              beam_runner_api_pb2.ArtifactStagingToRolePayload.FromString(\n                  dep.role_payload)).staged_name","sourceCodeStart":584,"sourceCodeEnd":620,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py#L584-L620","documentation":"Dataflow job staging requires a place to upload worker packages; neither --staging_location nor a GCS default could be resolved from options, so _stage_resources aborts before building the job description.","triggerScenarios":"Submitting a Dataflow job where staging_location was not set and the earlier __init__ defaulting (temp_location fallback) also left it None — typically reached via create_job_description.","commonSituations":"Older or unusual option-passing paths (e.g. constructing Job directly, template flows) that bypass the __init__ validation/defaulting; temp_location itself missing so no default could be applied.","solutions":["Set --staging_location=gs://<bucket>/<path> in pipeline options","Or ensure --temp_location is set, since __init__ defaults staging_location to temp_location","Use GoogleCloudOptions(staging_location='gs://...') programmatically"],"exampleFix":"# before\noptions = PipelineOptions(['--project=p', '--job_name=j'])\n# after\noptions = PipelineOptions([\n    '--project=p', '--job_name=j',\n    '--staging_location=gs://my-bucket/staging',\n    '--temp_location=gs://my-bucket/temp'])","handlingStrategy":"validation","validationCode":"gco = options.view_as(GoogleCloudOptions)\nif gco.staging_location is None and gco.temp_location is None:\n    raise SystemExit('set --staging_location or --temp_location')","typeGuard":"def has_staging(options) -> bool:\n    return options.view_as(GoogleCloudOptions).staging_location is not None","tryCatchPattern":null,"preventionTips":["Always set both staging_location and temp_location for Dataflow runs","Centralize GCS paths in one config module so jobs cannot be launched without them","Lint job-launch scripts for the presence of staging/temp flags"],"tags":["dataflow","configuration","gcs","staging"],"backgroundTag":"missing-required-flag","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}