What causes "error upgrading setuptools" in apache/beam?

Check network access to PyPI and configure proxy env vars if behind a corporate proxy

error upgrading setuptools

Error message

error upgrading setuptools

What it means

Raised in expansionx.SetUpPythonEnvironment when `pip install --upgrade setuptools` fails inside the Python expansion service virtualenv. Environment setup is aborted and no venv python path is returned.

Solutions

Check network access to PyPI and configure proxy env vars if behind a corporate proxy
Run `<venv>/bin/python -m pip install --upgrade setuptools` manually to see the underlying output
Pin a setuptools version compatible with the interpreter if a wheel resolution error occurs
Free disk space and confirm the venv directory is writable

Example fix

// before
// error upgrading setuptools
// after: pin compatible version
./venv/bin/python -m pip install "setuptools<60"

Defensive patterns

Strategy: validation

Validate before calling

if err := exec.Command(venvPython, "-m", "pip", "install", "--upgrade", "setuptools").Run(); err != nil {
    return fmt.Errorf("setuptools upgrade failed: %w", err)
}

Try / catch

python, err := expansionx.SetUpPythonEnvironment(ctx, py, pkg, extras)
if err != nil && strings.Contains(err.Error(), "error upgrading setuptools") {
    return fmt.Errorf("pip/setuptools bootstrap failed; check network and Python compatibility: %w", err)
}

Prevention

Ensure network access to PyPI and enough disk space
Use a Python version with prebuilt setuptools wheels
Pre-provision the venv manually once, then reuse the cached environment

When it happens

Trigger: startPythonExpansionService -> SetUpPythonEnvironment where the setuptools upgrade returns non-zero, typically network/PyPI access problems or an incompatible wheel for the platform/Python version.

Common situations: No internet or blocked PyPI; very old/new Python with no compatible setuptools wheel; disk space exhaustion; corporate mirror rejecting the package.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f59b569ca1ffc88f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/xlangx/expansionx/download.go:471

	beamPackage := fmt.Sprintf("apache_beam[gcp,aws,azure,dataframe]==%s", core.SdkVersion)
	venvDir := filepath.Join(
		cacheDir, "venvs",
		fmt.Sprintf("py-%s-beam-%s-%s", py, core.SdkVersion, strings.Join(extraPackages, ";")),
	)
	venvPython := filepath.Join(venvDir, "bin", "python")

	if _, err := os.Stat(venvPython); err != nil {
		err := exec.Command(py, "-m", "venv", venvDir).Run()
		if err != nil {
			return "", errors.Wrap(err, "error creating new virtual environment for python expansion service")
		}
		err = exec.Command(venvPython, "-m", "pip", "install", "--upgrade", "pip").Run()
		if err != nil {
			return "", errors.Wrap(err, "error upgrading pip")
		}
		err = exec.Command(venvPython, "-m", "pip", "install", "--upgrade", "setuptools").Run()
		if err != nil {
			return "", errors.Wrap(err, "error upgrading setuptools")
		}
		err = exec.Command(venvPython, "-m", "pip", "install", beamPackage, "pyparsing==2.4.2").Run()
		if err != nil {
			return "", errors.Wrap(err, fmt.Sprintf("error installing beam package: %v", beamPackage))
		}
		if len(extraPackages) > 0 {
			cmd := []string{"-m", "pip", "install"}
			cmd = append(cmd, extraPackages...)
			err = exec.Command(venvPython, cmd...).Run()
			if err != nil {
				return "", errors.Wrap(err, "error installing dependencies")
			}
		}
		err = exec.Command(venvPython, "-c", "import apache_beam").Run()
		if err != nil {
			return "", errors.Wrap(err, "apache beam installation failed in virtualenv")
		}
	}

View on GitHub (pinned to 12126d8942)