kubernetes/kops · error

error removing file %s: %w

Error message

error removing file %s: %w

What it means

FSPath.RemoveAll deletes every file in the local-filesystem tree by first listing the tree with ReadTree and then calling Remove (os.Remove) on each path. This error wraps the underlying os.Remove failure for a specific file, so the developer sees both which file could not be deleted and the root OS-level reason (permissions, missing parent dir, etc.). It aborts the loop on the first failure, leaving earlier files deleted and remaining files untouched.

Source

Thrown at util/pkg/vfs/fs.go:207

func (p *FSPath) String() string {
	return p.Path()
}

func (p *FSPath) Remove(ctx context.Context) error {
	return os.Remove(p.location)
}

func (p *FSPath) RemoveAll(ctx context.Context) error {
	tree, err := p.ReadTree(ctx)
	if err != nil {
		return err
	}

	for _, filePath := range tree {
		err := filePath.Remove(ctx)
		if err != nil {
			return fmt.Errorf("error removing file %s: %w", filePath, err)
		}
	}

	return nil
}

func (p *FSPath) RemoveAllVersions(ctx context.Context) error {
	return p.Remove(ctx)
}

func (p *FSPath) PreferredHash() (*hashing.Hash, error) {
	return p.Hash(hashing.HashAlgorithmSHA256)
}

func (p *FSPath) Hash(a hashing.HashAlgorithm) (*hashing.Hash, error) {
	klog.V(2).Infof("hashing file %q", p.location)

	return a.HashFile(p.location)

View on GitHub (pinned to 4c8573c808)

Solutions

1. Inspect the wrapped %w error (os errno) to identify the cause: EACCES/EPERM means fix permissions on the state directory (chmod/chown or run as the owning user); ENOENT means the file is already gone and can be treated as done or ignored.
2. Check for concurrent processes (other kops runs, background jobs) holding or deleting files in the same state directory and serialize the deletion.
3. If the underlying issue is a directory that is not empty due to partial deletes, re-run RemoveAll — the tree listing is re-read each attempt, so a second run usually completes.
4. Verify the filesystem is writable and not mounted read-only / in a degraded state (dmesg, mount output) before retrying.

Example fix

// before
tree, _ := p.ReadTree(ctx)
for _, filePath := range tree {
    _ = filePath.Remove(ctx) // silent partial failures
}
// after
if err := p.RemoveAll(ctx); err != nil {
    var pe *fs.PathError
    if errors.As(err, &pe) && errors.Is(pe.Err, syscall.ENOENT) {
        return nil // already deleted concurrently
    }
    return err // permission or I/O problem needs real fixing
}

// Go has no pre-check that guarantees a future delete succeeds, but you can
// pre-verify writability of the tree root before deleting:
func canDelete(dir string) error {
    info, err := os.Stat(dir)
    if err != nil { return err }
    if !info.IsDir() { return fmt.Errorf("%s is not a directory", dir) }
    f, err := os.OpenFile(dir, os.O_RDONLY, 0)
    if err != nil { return err }
    return f.Close()
}

if err := path.RemoveAll(ctx); err != nil {
    var pe *fs.PathError
    if errors.As(err, &pe) {
        switch {
        case errors.Is(pe.Err, syscall.ENOENT):
            return nil // already gone; treat as success
        case errors.Is(pe.Err, syscall.EACCES), errors.Is(pe.Err, syscall.EPERM):
            return fmt.Errorf("permission denied deleting %s: run as the owner of the state dir", pe.Path)
        }
    }
    return err
}

When it happens

Trigger: Calling vfscontext-based FSPath.RemoveAll(ctx) (or higher-level cluster/state deletion helpers that use it) when an entry in the tree cannot be removed: the file was deleted concurrently, a parent directory was removed mid-iteration, the process lacks write permission on the parent directory, or the path is a non-empty directory that os.Remove refuses to delete.

Common situations: Running kops against a local/FS-backed state store where the state directory permissions changed (e.g. created by root via a container, later used by a non-root user); two kops processes racing to delete the same state; a state dir mounted read-only or on a dead NFS mount; empty-directory entries in the tree that os.Remove cannot delete because they still contain children.

Related errors

• reading file %q: %v
• error reading user provided private key %q: %v
• reading encryption config %v: %v
• error reading SSH public key %v: %v
• reading from stdin: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/79d36225687234c8. Report an issue: GitHub.