dagger/dagger · error

failed to stat lowerPath %q

Error message

failed to stat lowerPath %q

What it means

During whiteout creation for a delete, the applier Lstats the path in each lowerdir to decide whether a whiteout device is needed. If Lstat fails with anything other than ENOENT/ENOTDIR (which are expected 'not found' results), the error is wrapped with this message and the apply aborts.

Source

Thrown at engine/snapshots/diffapply_linux.go:275

	}
	ca.dstStat = nil

	if overwrite && a.createWhiteoutDelete && ca.srcStat.Mode&unix.S_IFMT == unix.S_IFDIR {
		ca.setOpaque = true
	}

	if deleteOnly && a.createWhiteoutDelete {
		var foundLower bool
		for _, lowerdir := range a.lowerdirs {
			lowerPath, err := safeJoin(lowerdir, ca.subPath)
			if err != nil {
				return false, errors.Wrapf(err, "failed to join lowerdir %q and subPath %q", lowerdir, ca.subPath)
			}
			if _, err := os.Lstat(lowerPath); err == nil {
				foundLower = true
				break
			} else if !errors.Is(err, unix.ENOENT) && !errors.Is(err, unix.ENOTDIR) {
				return false, errors.Wrapf(err, "failed to stat lowerPath %q", lowerPath)
			}
		}
		if foundLower {
			ca.kind = continuityfs.ChangeKindAdd
			if ca.srcStat == nil {
				ca.srcStat = &syscall.Stat_t{
					Mode: syscall.S_IFCHR,
					Rdev: unix.Mkdev(0, 0),
				}
				ca.srcPath = ""
			}
			return false, nil
		}
	}

	return deleteOnly, nil
}

View on GitHub (pinned to 82ba2681db)

Solutions

1. Fix permissions so the applying process can traverse every lowerdir path component
2. Check the health of the storage backing the lowerdirs (dmesg, remount) if EIO/ESTALE is reported
3. Verify all configured lowerdirs still exist and were not modified during the apply
4. Retry the operation; transient network-storage errors often resolve after remount

// before apply, ensure every lowerdir is traversable
for _, lower := range lowerdirs {
    if err := unix.Access(lower, unix.X_OK); err != nil {
        return fmt.Errorf("lowerdir %s not traversable: %w", lower, err)
    }
}

if err := applier.Apply(ctx, change); err != nil {
    if strings.Contains(err.Error(), "failed to stat lowerPath") {
        if errors.Is(err, unix.ESTALE) || errors.Is(err, unix.EIO) {
            // network storage hiccup: remount and retry
            return remountAndRetry(ctx, change)
        }
        return fmt.Errorf("lower layer unhealthy: %w", err)
    }
    return err
}

When it happens

Trigger: os.Lstat(lowerPath) returns EACCES (no search permission on a lowerdir component), ELOOP (symlink loop), EIO/ESTALE (lowerdir on unhealthy or network storage), or ENAMETOOLONG while probing lower layers.

Common situations: Overlay lowerdirs on NFS or another network filesystem returning ESTALE; restrictive permissions on lower layer directories; broken symlink loops in lower layers; lowerdir removed or renamed while a diff is being applied.

Related errors

• failed to stat base file during overlay diff
• stat removed path %s: %w
• stat modified path %s: %w
• stat %s: %w
• check existing module at %q: %w

AI-assisted analysis of dagger/dagger@82ba2681db (2026-09-05). Data as JSON: /api/errors/9bfe50093fbb93e0. Report an issue: GitHub.