apache/beam · error · RuntimeError

Full traceback: \n Pip install failed for package: \n…

Error message

Full traceback: {} \n Pip install failed for package: {} \n Output from execution of subprocess: {}

What it means

processes.call catches subprocess.CalledProcessError and, when the command looks like a pip invocation, raises this RuntimeError embedding the full traceback, the pip package argument, and pip's captured output.

Solutions

1. Inspect the embedded pip output in the exception for the real pip failure cause
2. Verify the package name/version exists and is compatible with the Python version
3. Ensure network/proxy access to PyPI (or use --index-url to an internal index)
4. Pre-install packages in the container/requirement file instead of runtime pip install

Example fix

// before
processes.call('pip install some-pkg==9.9')  # version doesn't exist
// after
processes.call('pip', 'install', 'some-pkg==1.2.3')  # verified available version

# check the package exists in the index first
import subprocess
r = subprocess.run(['pip', 'download', '--no-deps', '-d', tmpdir, 'pkg==1.2.3'], capture_output=True)
assert r.returncode == 0, r.stderr

try:
    processes.call('pip', 'install', 'pkg==1.2.3')
except RuntimeError as e:
    if 'Pip install failed' in str(e):
        logging.error('pip failed: %s', e)  # embedded output shows pip's cause
        raise SystemExit(1) from e
    raise

When it happens

Trigger: Calling processes.call('pip', ..., 'install', package, ...) (the check inspects args[0][2] == 'pip') and pip exits with a nonzero status, e.g. when shell/split-string form ['pip','install','pkg'] is used.

Common situations: Pip install failing due to no network access, nonexistent package version, missing build toolchain for sdists, no-PyPI restriction on locked-down runners, or no matching distribution for the platform.

Related errors

• Could not generate external transform wrappers due to error
• Could not generate protos due to error
• error installing beam package
• error installing dependencies
• error upgrading pip

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/19d996fc7e3693a6. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/utils/processes.py:56

if TYPE_CHECKING:
  call = subprocess.call
  check_call = subprocess.check_call
  check_output = subprocess.check_output
  Popen = subprocess.Popen

else:

  def call(*args, **kwargs):
    if force_shell:
      kwargs['shell'] = True
    try:
      out = subprocess.call(*args, **kwargs)
    except OSError as e:
      raise RuntimeError("Executable {} not found".format(args[0])) from e
    except subprocess.CalledProcessError as error:
      if isinstance(args, tuple) and (args[0][2] == "pip"):
        raise RuntimeError( \
          "Full traceback: {}\n Pip install failed for package: {} \
          \n Output from execution of subprocess: {}" \
          .format(traceback.format_exc(), args[0][6], error. output)) from error
      else:
        raise RuntimeError("Full trace: {}\
           \n Output of the failed child process: {} " \
          .format(traceback.format_exc(), error.output)) from error
    return out

  def check_call(*args, **kwargs):
    if force_shell:
      kwargs['shell'] = True
    try:
      out = subprocess.check_call(*args, **kwargs)
    except OSError as e:
      raise RuntimeError("Executable {} not found".format(args[0])) from e
    except subprocess.CalledProcessError as error:
      if isinstance(args, tuple) and (args[0][2] == "pip"):

View on GitHub (pinned to 12126d8942)