apache/beam · error · RuntimeError

The staging_location must be specified.

Error message

The staging_location must be specified.

What it means

stage_job_resources requires a staging_location (the remote/local directory where artifacts are uploaded). If it is None, this RuntimeError is raised before any file is staged. It is a programming/config error, since runners normally derive this from the pipeline's staging location option.

Source

Thrown at sdks/python/apache_beam/runners/portability/stager.py:435

        Stages job resources to staging_location.

        Args:
          resources: A list of tuples of local file paths and file names (no
            paths) to be used for staging resources.
          staging_location: Location to stage the file.

        Returns:
          A list of file names (no paths) for the resources staged. All the
          files are assumed to be staged at staging_location.

        Raises:
          RuntimeError: If files specified are not found or error encountered
          while trying to create the resources (e.g., build a setup package).
        """
    # Make sure that all required options are specified.
    if staging_location is None:
      raise RuntimeError('The staging_location must be specified.')

    staged_resources = []
    for file_path, staged_path, sha256 in resources:
      self.stage_artifact(
          file_path, FileSystems.join(staging_location, staged_path), sha256)
      staged_resources.append(staged_path)

    return staged_resources

  def create_and_stage_job_resources(
      self,
      options: PipelineOptions,
      build_setup_args: Optional[list[str]] = None,
      temp_dir: Optional[str] = None,
      pypi_requirements: Optional[list[str]] = None,
      populate_requirements_cache: Optional[Callable[[str, str, bool],
                                                     None]] = None,
      staging_location: Optional[str] = None):

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass staging_location explicitly to create_and_stage_job_resources/stage_job_resources
2. Set --staging_location (and --temp_location) in PipelineOptions
3. Ensure the runner's default staging path resolution is intact (e.g. valid GCS/ s3 project settings)

Example fix

// before
stager.create_and_stage_job_resources(setup_options)
// after
stager.create_and_stage_job_resources(setup_options, staging_location='gs://bucket/staging')

staging_location = options.view_as(GoogleCloudOptions).staging_location
if not staging_location:
    raise SystemExit('--staging_location/--temp_location must be set')

def has_staging_location(opts): return bool(getattr(opts, 'staging_location', None))

try:
    stager.stage_job_resources(resources, staging_location)
except RuntimeError as e:
    if 'staging_location must be specified' in str(e):
        sys.exit('Pass --staging_location or set it in options')
    raise

When it happens

Trigger: Calling stage_job_resources (or create_and_stage_job_resources) without staging_location and without the options needed to derive it.

Common situations: Directly invoking Stager in custom tooling and forgetting staging_location; pipeline options missing --staging_location / --temp_location so the default resolution yields None.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• The --sdk_location option was used with an unsupported type
• Need to set the filepattern of a TFRecordIO.Read transform
• projectId can't be empty
• JDBC URL cannot be blank
• If JDBC type is not specified, then Driver Class Name and Dr

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8665e11370fb6ac5. Report an issue: GitHub.