{"record":{"id":"62e6a9509117e462","repo":"argoproj/argo-workflows","slug":"failed-to-stat-artifact-path-q-at-s-w","errorCode":null,"errorMessage":"failed to stat artifact path %q at %s: %w","messagePattern":"failed to stat artifact path %q at (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/argoexec/commands/emissary.go","lineNumber":409,"sourceCode":"\t\t\treturn fmt.Errorf(\"failed to stat input artifact %q at %s: %w\", art.Name, src, statErr)\n\t\t}\n\t\tdst := art.Path\n\t\tif dst == \"\" {\n\t\t\tcontinue\n\t\t}\n\t\tif parent := filepath.Dir(dst); parent != \"\" && parent != \"/\" {\n\t\t\tif err := os.MkdirAll(parent, 0o755); err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to create parent directory for artifact %q at %s: %w\", art.Name, dst, err)\n\t\t\t}\n\t\t}\n\t\t// If nothing exists at art.Path, just create the symlink. Creating is\n\t\t// always safe — os.Symlink returns EEXIST rather than overwriting and the\n\t\t// MkdirAll above only ever creates — so even when art.Path resolves into a\n\t\t// user volume we deliberately let the artifact land there (the user asked\n\t\t// for it). Only an *overwrite* can destroy data, and that is gated below.\n\t\tif _, err := os.Lstat(dst); err != nil {\n\t\t\tif !os.IsNotExist(err) {\n\t\t\t\treturn fmt.Errorf(\"failed to stat artifact path %q at %s: %w\", art.Name, dst, err)\n\t\t\t}\n\t\t} else {\n\t\t\t// Something is already at art.Path. Replacing it (os.RemoveAll then\n\t\t\t// symlink) reproduces the legacy SubPath mount's shadowing — but only\n\t\t\t// when it is safe. RemoveAll resolves symlinks in the parent chain, so\n\t\t\t// resolve the parent to find where the delete would actually land\n\t\t\t// (resolve the parent, not the final element, so an image symlink *at*\n\t\t\t// art.Path is just unlinked rather than followed). If that resolved\n\t\t\t// path overlaps a user-declared volume, clearing it would recurse into\n\t\t\t// and destroy a live PVC/hostPath/emptyDir, so refuse. Benign system\n\t\t\t// mounts (tmpfs /run, the overlay rootfs) are not declared user volumes\n\t\t\t// and so remain safe to shadow.\n\t\t\trealParent, evalErr := filepath.EvalSymlinks(filepath.Dir(dst))\n\t\t\tif evalErr != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to resolve parent of artifact path %q at %s: %w\", art.Name, dst, evalErr)\n\t\t\t}\n\t\t\tresolved := filepath.Join(realParent, filepath.Base(dst))\n\t\t\tif mnt := common.FindOverlappingVolume(tmpl, resolved); mnt != nil {","sourceCodeStart":391,"sourceCodeEnd":427,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/cmd/argoexec/commands/emissary.go#L391-L427","documentation":"After ensuring the parent directory exists, the executor Lstats the artifact destination (art.Path) to decide whether a safe symlink create suffices or a guarded overwrite is needed. A stat error that is NOT 'does not exist' is fatal and wrapped here — the executor cannot determine what currently sits at the destination, so it refuses to proceed rather than risk destroying data with a RemoveAll.","triggerScenarios":"os.Lstat(dst) fails with something other than ENOENT: EACCES on the destination directory, ELOOP from a symlink cycle in the path, or EIO on the underlying volume while probing the destination path.","commonSituations":"Artifact paths on volumes with restrictive permissions (executor can list parent but not lstat children); symlink loops in user-mounted volumes; flaky network storage (NFS/EFS) returning transient I/O errors; destination inside a containerd subPath mount that is mid-unmount.","solutions":["Inspect the wrapped errno: EACCES → grant the executor user read/execute on the destination directory chain; ELOOP → fix the symlink cycle.","For NFS/network volumes, retry the workflow — transient EIO often resolves.","Ensure artifact.path's parent chain consists of real directories, not symlink loops.","Match volume permissions with securityContext fsGroup so the executor can traverse the path.","If the path should simply not exist, pre-clean it in the pod setup so Lstat returns ENOENT and the plain create path is used."],"exampleFix":"// before (EACCES on traversal)\nvolumeMount:\n  name: data\n  mountPath: /data\n# dir mode 0700 owned by 1000; executor runs as root... or vice versa\n// after\nchmod o+rx /data   # or set pod fsGroup so the executor can traverse","handlingStrategy":"try-catch","validationCode":"// Pre-flight: destination must be statable or nonexistent:\nif [ -e '<artifact.path>' ] && [ ! -r \"$(dirname '<artifact.path>')\" ]; then\n  echo \"cannot inspect $(dirname '<artifact.path>')\" >&2; exit 1\nfi","typeGuard":null,"tryCatchPattern":"try {\n  await stageArtifacts(tmpl)\n} catch (e) {\n  if (String(e).includes('failed to stat artifact path')) {\n    // fix traversal permissions or symlink loops; retry on transient EIO\n  }\n}","preventionTips":["Keep artifact destination directory chains free of symlink loops.","Grant traverse (x) permission on every directory component of artifact.path.","Avoid flaky network filesystems for artifact destinations, or add retries.","Pre-create/clean destination paths in pod setup so the safe create path runs."],"tags":["kubernetes","argo-workflows","artifacts","filesystem","symlink"],"backgroundTag":"stat-failed","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}