apache/beam · error

error installing dependencies

Error message

error installing dependencies

What it means

Raised in expansionx.SetUpPythonEnvironment when installing the caller-supplied extra packages into the expansion service virtualenv fails. These are the dependencies the user requested alongside apache_beam for the transform being expanded.

Solutions

1. Verify each extra package name and version pin is correct and resolvable on the configured index
2. Install the extras manually in the venv to surface the concrete pip error
3. Add a private index (--extra-index-url / PIP_EXTRA_INDEX_URL) if packages come from an internal registry
4. Relax conflicting pins or move heavy native deps to wheels compatible with the platform

Example fix

// before
xlangx.SetUpPythonEnvironment(ctx, "python3", pyPackage, []string{"mypkg==9.9.9"}) // nonexistent pin
// after
xlangx.SetUpPythonEnvironment(ctx, "python3", pyPackage, []string{"mypkg==1.2.0"})

for _, p := range extraPackages {
    name, _, _ := strings.Cut(p, "==")
    if !knownOnIndex(name) {
        return fmt.Errorf("extra package %q not found on configured index", p)
    }
}

python, err := expansionx.SetUpPythonEnvironment(ctx, py, pkg, extras)
if err != nil && strings.Contains(err.Error(), "error installing dependencies") {
    return fmt.Errorf("extra packages %v failed to install; verify names/versions and index access: %w", extras, err)
}

When it happens

Trigger: startPythonExpansionService -> SetUpPythonEnvironment with non-empty extraPackages where `pip install <extraPackages...>` returns non-zero: a typo'd or nonexistent package name, an unresolvable version pin, or no network.

Common situations: Extra packages requiring compilation toolchains (e.g. numpy/pandas on odd platforms); private packages not on public PyPI; version pins conflicting with apache_beam's requirements; proxy-blocked downloads.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

• Class name must not be empty
• Constructor or constructor method can only be specified once
• Could not find coder for URN " + urn
• Dependencies must be a list of strings, got
• error creating new virtual environment for python expansion…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4eb2c36666d12eae. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/xlangx/expansionx/download.go:482

		}
		err = exec.Command(venvPython, "-m", "pip", "install", "--upgrade", "pip").Run()
		if err != nil {
			return "", errors.Wrap(err, "error upgrading pip")
		}
		err = exec.Command(venvPython, "-m", "pip", "install", "--upgrade", "setuptools").Run()
		if err != nil {
			return "", errors.Wrap(err, "error upgrading setuptools")
		}
		err = exec.Command(venvPython, "-m", "pip", "install", beamPackage, "pyparsing==2.4.2").Run()
		if err != nil {
			return "", errors.Wrap(err, fmt.Sprintf("error installing beam package: %v", beamPackage))
		}
		if len(extraPackages) > 0 {
			cmd := []string{"-m", "pip", "install"}
			cmd = append(cmd, extraPackages...)
			err = exec.Command(venvPython, cmd...).Run()
			if err != nil {
				return "", errors.Wrap(err, "error installing dependencies")
			}
		}
		err = exec.Command(venvPython, "-c", "import apache_beam").Run()
		if err != nil {
			return "", errors.Wrap(err, "apache beam installation failed in virtualenv")
		}
	}
	return venvPython, nil
}

View on GitHub (pinned to 12126d8942)