{"record":{"id":"6a25b3f7ec5b20ee","repo":"apache/beam","slug":"error-creating-new-virtual-environment-for-python-expansion","errorCode":null,"errorMessage":"error creating new virtual environment for python expansion service","messagePattern":"error creating new virtual environment for python expansion service","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/core/runtime/xlangx/expansionx/download.go","lineNumber":463,"sourceCode":"\t}\n\textraPackages := []string{}\n\tif len(extraPackage) > 0 {\n\t\textraPackages = strings.Split(extraPackage, \" \")\n\t}\n\n\t// create python virtual environment\n\tsort.Strings(extraPackages)\n\tbeamPackage := fmt.Sprintf(\"apache_beam[gcp,aws,azure,dataframe]==%s\", core.SdkVersion)\n\tvenvDir := filepath.Join(\n\t\tcacheDir, \"venvs\",\n\t\tfmt.Sprintf(\"py-%s-beam-%s-%s\", py, core.SdkVersion, strings.Join(extraPackages, \";\")),\n\t)\n\tvenvPython := filepath.Join(venvDir, \"bin\", \"python\")\n\n\tif _, err := os.Stat(venvPython); err != nil {\n\t\terr := exec.Command(py, \"-m\", \"venv\", venvDir).Run()\n\t\tif err != nil {\n\t\t\treturn \"\", errors.Wrap(err, \"error creating new virtual environment for python expansion service\")\n\t\t}\n\t\terr = exec.Command(venvPython, \"-m\", \"pip\", \"install\", \"--upgrade\", \"pip\").Run()\n\t\tif err != nil {\n\t\t\treturn \"\", errors.Wrap(err, \"error upgrading pip\")\n\t\t}\n\t\terr = exec.Command(venvPython, \"-m\", \"pip\", \"install\", \"--upgrade\", \"setuptools\").Run()\n\t\tif err != nil {\n\t\t\treturn \"\", errors.Wrap(err, \"error upgrading setuptools\")\n\t\t}\n\t\terr = exec.Command(venvPython, \"-m\", \"pip\", \"install\", beamPackage, \"pyparsing==2.4.2\").Run()\n\t\tif err != nil {\n\t\t\treturn \"\", errors.Wrap(err, fmt.Sprintf(\"error installing beam package: %v\", beamPackage))\n\t\t}\n\t\tif len(extraPackages) > 0 {\n\t\t\tcmd := []string{\"-m\", \"pip\", \"install\"}\n\t\t\tcmd = append(cmd, extraPackages...)\n\t\t\terr = exec.Command(venvPython, cmd...).Run()\n\t\t\tif err != nil {","sourceCodeStart":445,"sourceCodeEnd":481,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/core/runtime/xlangx/expansionx/download.go#L445-L481","documentation":"Raised in expansionx.SetUpPythonEnvironment when running `python -m venv <venvDir>` fails while preparing a local Python expansion service. It means a virtualenv could not be created, so no expansion service environment exists. The wrapped error comes from exec.Command(...).Run().","triggerScenarios":"startPythonExpansionService -> SetUpPythonEnvironment when the chosen python interpreter lacks the venv module, the venv target path is unwritable/exists corrupt, or the python binary itself is broken.","commonSituations":"System python installed without python3-venv (common on Debian/Ubuntu); pointing at a python without ensurepip; disk full or permission denied in the cache directory; corrupted venv left behind by a previous failed run.","solutions":["Install venv support for the interpreter (e.g. apt install python3-venv / python3-pip)","Delete any stale/corrupt venv directory in the expansionx cache path and retry","Verify `python -m venv /tmp/testenv` works manually with the same interpreter","Check disk space and write permissions for the directory holding the venv"],"exampleFix":"// before\n// error creating new virtual environment...\n// after (fix host env, not code)\nsudo apt-get install -y python3-venv python3-pip\nrm -rf ~/.cache/beam/expansion_service_venv","handlingStrategy":"validation","validationCode":"if err := exec.Command(py, \"-m\", \"venv\", \"/tmp/beam_venv_probe\").Run(); err != nil {\n    return fmt.Errorf(\"python %q cannot create venvs: %w (install python3-venv?)\", py, err)\n}\nos.RemoveAll(\"/tmp/beam_venv_probe\")","typeGuard":null,"tryCatchPattern":"python, err := expansionx.SetUpPythonEnvironment(ctx, py, pkg, extras)\nif err != nil {\n    return fmt.Errorf(\"venv creation failed; check python3-venv, disk space, and permissions: %w\", err)\n}","preventionTips":["Install python3-venv/ensurepip for the interpreter you pass","Pre-create a writable cache directory and keep free disk space","Delete stale venv directories from failed runs"],"tags":["python","virtualenv","environment-setup","cross-language"],"backgroundTag":"command-not-found","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}