apache/beam · error · ValueError

Could not find Python executable.

Error message

Could not find Python executable.

What it means

Raised by Stager._get_python_executable when neither the BEAM_PYTHON environment variable nor sys.executable yields a usable Python binary. It ensures dependency installation during staging runs with a valid interpreter. Since sys.executable is normally set, this almost always means BEAM_PYTHON was set to an empty or invalid value.

Solutions

1. Unset or correctly set the BEAM_PYTHON environment variable to a valid python binary path
2. Ensure the pipeline is launched from a normal Python interpreter where sys.executable is populated
3. Verify the BEAM_PYTHON path exists and is executable (e.g. BEAM_PYTHON=$(which python3))

Example fix

# before
export BEAM_PYTHON=""
# after
export BEAM_PYTHON=$(which python3)

import os, sys
python_bin = os.environ.get('BEAM_PYTHON') or sys.executable
if not python_bin:
    raise SystemExit('Set BEAM_PYTHON to a valid python binary before launching.')

def has_python_executable() -> bool:
    import shutil
    bin_ = os.environ.get('BEAM_PYTHON') or sys.executable
    return bool(bin_) and (shutil.which(bin_) is not None or os.path.isfile(bin_))

try:
    stage_dependencies(...)
except ValueError as e:
    if 'Could not find Python executable' in str(e):
        os.environ['BEAM_PYTHON'] = sys.executable or shutil.which('python3')
    raise

When it happens

Trigger: Setting BEAM_PYTHON to an empty string (or otherwise falsy) while sys.executable is also empty/None, e.g. running embedded in an interpreter without a resolved executable.

Common situations: BEAM_PYTHON='' exported in CI or Docker images; running under embedded Python or frozen binaries where sys.executable is empty.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors

• Incompatible environments
• Java is not correctly installed in JAVA_HOME=
• Java must be installed on this system to use this…
• No proto encoding for PaneInfoCoder, always part of…
• no python installation found. If you use a custom container…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c68d348b8827955a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/portability/stager.py:690

    with open(os.path.join(temp_dir, EXTRA_PACKAGES_FILE), 'wt') as f:
      for package in local_packages:
        f.write('%s\n' % os.path.basename(package))
    # Note that the caller of this function is responsible for deleting the
    # temporary folder where all temp files are created, including this one.
    resources.append(
        Stager._create_file_stage_to_artifact(
            os.path.join(temp_dir, EXTRA_PACKAGES_FILE), EXTRA_PACKAGES_FILE))

    return resources

  @staticmethod
  def _get_python_executable():
    # Allow overriding the python executable to use for downloading and
    # installing dependencies, otherwise use the python executable for
    # the current process.
    python_bin = os.environ.get('BEAM_PYTHON') or sys.executable
    if not python_bin:
      raise ValueError('Could not find Python executable.')
    return python_bin

  @staticmethod
  def _remove_dependency_from_requirements(
      requirements_file: str, dependency_to_remove: str, temp_directory_path):
    """Function to remove dependencies from a given requirements file."""
    # read all the dependency names
    with open(requirements_file, 'r') as f:
      lines = f.readlines()

    tmp_requirements_filename = os.path.join(
        temp_directory_path, 'tmp_requirements.txt')

    with open(tmp_requirements_filename, 'w') as tf:
      for i in range(len(lines)):
        if not lines[i].startswith(dependency_to_remove):
          tf.write(lines[i])

View on GitHub (pinned to 12126d8942)