apache/beam · error · RuntimeError

The file %s cannot be found. It was specified in the --exper

Error message

The file %s cannot be found. It was specified in the --experiment='jar_packages=' command line option.

What it means

After validating the .jar suffix, _create_jar_packages checks each package exists locally; remote paths are downloaded first. If a non-remote jar is not an existing file, this RuntimeError is raised naming the missing package.

Source

Thrown at sdks/python/apache_beam/runners/portability/stager.py:586

    resources: list[beam_runner_api_pb2.ArtifactInformation] = []
    staging_temp_dir = tempfile.mkdtemp(dir=temp_dir)
    local_packages: list[str] = []
    for package in jar_packages:
      if not os.path.basename(package).endswith('.jar'):
        raise RuntimeError(
            'The --experiment=\'jar_packages=\' option expects a full path '
            'ending with ".jar" instead of %s' % package)

      if not os.path.isfile(package):
        if Stager._is_remote_path(package):
          # Download remote package.
          _LOGGER.info(
              'Downloading jar package: %s locally before staging', package)
          _, last_component = FileSystems.split(package)
          local_file_path = FileSystems.join(staging_temp_dir, last_component)
          Stager._download_file(package, local_file_path)
        else:
          raise RuntimeError(
              'The file %s cannot be found. It was specified in the '
              '--experiment=\'jar_packages=\' command line option.' % package)
      else:
        local_packages.append(package)

    local_packages.extend([
        FileSystems.join(staging_temp_dir, f)
        for f in os.listdir(staging_temp_dir)
    ])

    for package in local_packages:
      basename = os.path.basename(package)
      resources.append(Stager._create_file_stage_to_artifact(package, basename))

    return resources

  @staticmethod
  def _create_extra_packages(

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass the absolute path to the jar that exists on the machine launching the job
2. Host the jar at an https:// URL so Beam downloads it during staging
3. Rebuild/copy the jar to the driver environment before submission
4. Verify with `ls -l <jar>` on the submission host

Example fix

// before
--experiment=jar_packages=target/my-lib.jar
// after
--experiment=jar_packages=/home/user/project/target/my-lib.jar

packages = options.view_as(SetupOptions).jar_packages or []
missing = [p for p in packages
           if not p.startswith(('http://', 'https://', 'gs://'))
           and not os.path.isfile(p)]
if missing:
    raise SystemExit(f'jar_packages files missing: {missing}')

def jar_exists_or_remote(p):
    return Stager._is_remote_path(p) or os.path.isfile(p)

try:
    stager.create_and_stage_job_resources(setup_options)
except RuntimeError as e:
    if "jar_packages=' command line option" in str(e):
        sys.exit(f'Jar not found on driver host: {e}')
    raise

When it happens

Trigger: create_job_resources with a jar_packages entry that is not a remote path and os.path.isfile(package) is False.

Common situations: Jar built locally but path built on another host (Dataflow/containers don't see your laptop's /home path); deleted jar; typo in path; using a relative path from a different working directory.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• The file %s cannot be found. It was specified in the --setup
• The file "%s" cannot be found. Its location was specified by
• The --setup_file option expects the full path to a file name
• The --sdk_location option was used with an unsupported type
• The --experiment='jar_packages=' option expects a full path

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d2b358b59861b225. Report an issue: GitHub.