{"record":{"id":"b6544de878d8b4ba","repo":"argoproj/argo-workflows","slug":"failed-to-leave-working-directory-before-staging-i","errorCode":null,"errorMessage":"failed to leave working directory before staging input artifacts: %w","messagePattern":"failed to leave working directory before staging input artifacts: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/argoexec/commands/emissary.go","lineNumber":353,"sourceCode":"func stageInputArtifacts(ctx context.Context, tmpl *wfv1.Template) error {\n\treturn stageInputArtifactsAt(ctx, common.ExecutorArtifactBaseDir, tmpl)\n}\n\n// stageInputArtifactsAt is the parameterized form used by tests; production\n// calls stageInputArtifacts with the constants. It links each input artifact\n// into place and re-enters the working directory afterwards, stepping off it\n// for the duration: linking replaces the cwd when an artifact's path is the\n// container's workingDir, Windows refuses to delete a directory in use as a\n// working directory, and a child forked with the deleted directory as cwd\n// would see getcwd() fail and relative paths resolve to nothing. The final\n// chdir follows the symlink to whatever now sits at the path.\nfunc stageInputArtifactsAt(ctx context.Context, baseDir string, tmpl *wfv1.Template) error {\n\torigWd, err := os.Getwd()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read working directory before staging input artifacts: %w\", err)\n\t}\n\tif err := os.Chdir(varRunArgo); err != nil {\n\t\treturn fmt.Errorf(\"failed to leave working directory before staging input artifacts: %w\", err)\n\t}\n\tif err := linkInputArtifactsAt(ctx, baseDir, tmpl); err != nil {\n\t\treturn err\n\t}\n\tif err := os.Chdir(origWd); err != nil {\n\t\treturn fmt.Errorf(\"failed to re-enter working directory %q after staging input artifacts (an input artifact staged at the workingDir path must be a directory): %w\", origWd, err)\n\t}\n\treturn nil\n}\n\n// linkInputArtifactsAt creates a symlink at each input artifact's path\n// pointing to the file that supervisor wrote under /argo/inputs/artifacts/\n// <name>. This replaces the legacy SubPath bind-mount-per-artifact scheme,\n// which can't be used in init-less mode because kubelet pre-creates SubPath\n// entries as empty directories before supervisor can write the real file.\n//\n// Behavior notes for workflow authors: in init-less mode art.Path is a\n// symlink rather than a regular file. `cat`, `open()`, `tar`, `cp`,","sourceCodeStart":335,"sourceCodeEnd":371,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/cmd/argoexec/commands/emissary.go#L335-L371","documentation":"To stage input artifacts safely, argoexec temporarily chdirs to /var/run/argo (so symlinks can be created without a deleted-cwd problem) and later chdirs back to the original directory. This error means the temporary chdir into /var/run/argo failed, so artifact staging is aborted before anything was linked.","triggerScenarios":"os.Chdir(varRunArgo) returns an error — the /var/run/argo directory does not exist (volume mount removed/renamed) or the process lacks execute permission on it (restricted pod security standards, read-only tmpfs).","commonSituations":"Custom pod specs that mount /var/run/argo read-only; hardened securityContext (no root, restricted filesystem) blocking access; controller/executor version mismatch where the emptyDir was never created; running argoexec locally where /var/run/argo doesn't exist.","solutions":["Verify /var/run/argo exists and is writable inside the executor container.","Check the pod's securityContext and any PSA/OPA policies restricting filesystem writes.","Ensure the pod spec includes the standard /var/run/argo emptyDir volume.","Read the wrapped errno (ENOENT vs EACCES vs EROFS) to pinpoint the cause.","Use the stock executor pod spec from the installed Argo version instead of a hand-modified one."],"exampleFix":"// before (read-only mount in custom podspec)\n- name: argo-run\n  emptyDir: {}\n  # mounted readOnly: true\n// after\n- name: argo-run\n  emptyDir: {}   # mounted normally (writable)","handlingStrategy":"validation","validationCode":"// Verify /var/run/argo is writable before staging:\ntest -d /var/run/argo && test -w /var/run/argo || { echo \"/var/run/argo not writable\" >&2; exit 1; }","typeGuard":null,"tryCatchPattern":"try {\n  await stageArtifacts(tmpl)\n} catch (e) {\n  if (String(e).includes('failed to leave working directory')) {\n    // check securityContext / volume mount readOnly flags\n  }\n}","preventionTips":["Keep the standard /var/run/argo emptyDir writable in all pod mutations.","Audit mutating webhooks/PSAs that add readOnly mounts.","Test templates with restricted securityContext images early."],"tags":["kubernetes","argo-workflows","filesystem","permissions"],"backgroundTag":"permission-denied","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}