apache/beam · error · RuntimeError

Cannot find default Beam SDK tar file

Error message

Cannot find default Beam SDK tar file "%s"

What it means

When --sdk_location is left at 'default', Beam expects a locally installed SDK tarball (apache_beam-*.tar.gz in the installed package directory). This RuntimeError is raised when that default tarball file cannot be found on disk.

Solutions

1. Install a packaged Beam release (pip install apache_beam) so the SDK tarball is present
2. Set --sdk_location explicitly to a local apache_beam-*.tar.gz or a https:// URL to one from PyPI
3. Set --sdk_location='' to skip staging the SDK when a prebuilt SDK container image is used (--prebuild_sdk_container_engine or environment_type=DOCKER/PROCESS)

import apache_beam, os, glob
sdks = glob.glob(os.path.join(os.path.dirname(apache_beam.__file__), 'apache_beam-*.tar.gz'))
if not sdks:
    options.sdk_location = 'https://files.pythonhosted.org/...'  # or prebuild container

def default_sdk_tarball_exists():
    import apache_beam, glob, os
    return bool(glob.glob(os.path.join(os.path.dirname(apache_beam.__file__), '*.tar.gz')))

try:
    stager.create_and_stage_job_resources(setup_options)
except RuntimeError as e:
    if 'Cannot find default Beam SDK tar file' in str(e):
        sys.exit('Install a packaged apache_beam release or set --sdk_location / prebuild SDK container')
    raise

When it happens

Trigger: create_job_resources with sdk_location == 'default' and the expected default sdk_path does not exist (os.path.exists false).

Common situations: Beam installed from a git checkout or via a mechanism that did not ship the source tarball (e.g. some pip installs, conda, or slim containers); running with an editable install lacking the built tarball.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Artifact not found at
• Coder registry has no fallback coder. This can happen if…
• Could not find python-snappy, google-crc32c, or crcmod. To…
• custom_gcs_temp_location must be str or ValueProvider
• Delimiter must be a non-empty bytes sequence.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bda22174d770ea9f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/portability/stager.py:366

          # Equivalent to 'default' now, leaving for backwards compatibility.
          pass
        else:
          if os.path.isdir(setup_options.sdk_location):
            sdk_path = os.path.join(
                setup_options.sdk_location, names.STAGED_SDK_SOURCES_FILENAME)
          else:
            sdk_path = setup_options.sdk_location

          if os.path.isfile(sdk_path):
            _LOGGER.info('Copying Beam SDK "%s" to staging location.', sdk_path)
            resources.append(
                Stager._create_file_stage_to_artifact(
                    sdk_path,
                    Stager._desired_sdk_filename_in_staging_location(
                        setup_options.sdk_location)))
          else:
            if setup_options.sdk_location == 'default':
              raise RuntimeError(
                  'Cannot find default Beam SDK tar file "%s"' % sdk_path)
            elif not setup_options.sdk_location:
              _LOGGER.info(
                  'Beam SDK will not be staged since --sdk_location '
                  'is empty.')
            else:
              raise RuntimeError(
                  'The file "%s" cannot be found. Its location was specified '
                  'by the --sdk_location command-line option.' % sdk_path)
    # The following artifacts are not processed by python sdk container boot
    # sequence in a setup mode and hence should not be skipped even if a
    # prebuilt sdk container image is used.

    # TODO(heejong): remove jar_packages experimental flag when cross-language
    #   dependency management is implemented for all runners.
    # Handle jar packages that should be staged for Java SDK Harness.
    jar_packages = options.view_as(DebugOptions).lookup_experiment(
        'jar_packages')

View on GitHub (pinned to 12126d8942)