{"record":{"id":"9bfe50093fbb93e0","repo":"dagger/dagger","slug":"failed-to-stat-lowerpath-q","errorCode":null,"errorMessage":"failed to stat lowerPath %q","messagePattern":"failed to stat lowerPath %q","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"engine/snapshots/diffapply_linux.go","lineNumber":275,"sourceCode":"\t}\n\tca.dstStat = nil\n\n\tif overwrite && a.createWhiteoutDelete && ca.srcStat.Mode&unix.S_IFMT == unix.S_IFDIR {\n\t\tca.setOpaque = true\n\t}\n\n\tif deleteOnly && a.createWhiteoutDelete {\n\t\tvar foundLower bool\n\t\tfor _, lowerdir := range a.lowerdirs {\n\t\t\tlowerPath, err := safeJoin(lowerdir, ca.subPath)\n\t\t\tif err != nil {\n\t\t\t\treturn false, errors.Wrapf(err, \"failed to join lowerdir %q and subPath %q\", lowerdir, ca.subPath)\n\t\t\t}\n\t\t\tif _, err := os.Lstat(lowerPath); err == nil {\n\t\t\t\tfoundLower = true\n\t\t\t\tbreak\n\t\t\t} else if !errors.Is(err, unix.ENOENT) && !errors.Is(err, unix.ENOTDIR) {\n\t\t\t\treturn false, errors.Wrapf(err, \"failed to stat lowerPath %q\", lowerPath)\n\t\t\t}\n\t\t}\n\t\tif foundLower {\n\t\t\tca.kind = continuityfs.ChangeKindAdd\n\t\t\tif ca.srcStat == nil {\n\t\t\t\tca.srcStat = &syscall.Stat_t{\n\t\t\t\t\tMode: syscall.S_IFCHR,\n\t\t\t\t\tRdev: unix.Mkdev(0, 0),\n\t\t\t\t}\n\t\t\t\tca.srcPath = \"\"\n\t\t\t}\n\t\t\treturn false, nil\n\t\t}\n\t}\n\n\treturn deleteOnly, nil\n}\n","sourceCodeStart":257,"sourceCodeEnd":293,"githubUrl":"https://github.com/dagger/dagger/blob/82ba2681dbe30d3547a1dc50ea495900ab5b6047/engine/snapshots/diffapply_linux.go#L257-L293","documentation":"During whiteout creation for a delete, the applier Lstats the path in each lowerdir to decide whether a whiteout device is needed. If Lstat fails with anything other than ENOENT/ENOTDIR (which are expected 'not found' results), the error is wrapped with this message and the apply aborts.","triggerScenarios":"os.Lstat(lowerPath) returns EACCES (no search permission on a lowerdir component), ELOOP (symlink loop), EIO/ESTALE (lowerdir on unhealthy or network storage), or ENAMETOOLONG while probing lower layers.","commonSituations":"Overlay lowerdirs on NFS or another network filesystem returning ESTALE; restrictive permissions on lower layer directories; broken symlink loops in lower layers; lowerdir removed or renamed while a diff is being applied.","solutions":["Fix permissions so the applying process can traverse every lowerdir path component","Check the health of the storage backing the lowerdirs (dmesg, remount) if EIO/ESTALE is reported","Verify all configured lowerdirs still exist and were not modified during the apply","Retry the operation; transient network-storage errors often resolve after remount"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// before apply, ensure every lowerdir is traversable\nfor _, lower := range lowerdirs {\n    if err := unix.Access(lower, unix.X_OK); err != nil {\n        return fmt.Errorf(\"lowerdir %s not traversable: %w\", lower, err)\n    }\n}","typeGuard":null,"tryCatchPattern":"if err := applier.Apply(ctx, change); err != nil {\n    if strings.Contains(err.Error(), \"failed to stat lowerPath\") {\n        if errors.Is(err, unix.ESTALE) || errors.Is(err, unix.EIO) {\n            // network storage hiccup: remount and retry\n            return remountAndRetry(ctx, change)\n        }\n        return fmt.Errorf(\"lower layer unhealthy: %w\", err)\n    }\n    return err\n}","preventionTips":["Keep lowerdirs on healthy local storage when possible","Grant the applying process search (x) permission on all lowerdir components","Do not remove or rename lowerdirs while a diff apply is running","Treat ESTALE/EIO as a storage health signal, not a diff problem"],"tags":["filesystem","overlayfs","stat","lowerdir"],"backgroundTag":"snapshot-apply-failed","analyzedSha":"82ba2681dbe30d3547a1dc50ea495900ab5b6047","analyzedAt":"2026-09-05T07:21:37.930Z","contentChangedAt":"2026-09-05T07:21:37.930Z","schemaVersion":2},"datasetVersion":"2026-09-12T12:17:11.808Z"}