apache/beam · error

failed to delete: (remove: )

Error message

failed to delete: %v (remove: %v)

What it means

Returned by artifact.retrieve when an existing file at the destination path cannot be removed before re-downloading the artifact. The library deletes a previously retrieved file so it can rewrite it cleanly; os.Remove failed and the filename plus underlying error are embedded in the message.

Solutions

1. Ensure dest points to an empty writable directory (e.g. a fresh temp dir) used only for artifact retrieval.
2. Fix permissions on the destination directory so the process user can delete files in it (chmod/chown the dir).
3. If the target path is a directory, remove the stale directory before rerunning Materialize.
4. Run the worker with sufficient privileges or remount the volume read-write.

Example fix

// before: shared, possibly dirty dest
artifact.Materialize(ctx, endpoint, deps, rt, "/opt/beam/artifacts")
// after: fresh writable staging dir per run
dest, _ := os.MkdirTemp("", "beam-artifacts-")
artifact.Materialize(ctx, endpoint, deps, rt, dest)

func ensureWritableCleanDir(dest string) error {
	info, err := os.Stat(dest)
	if err != nil {
		return err
	}
	if !info.IsDir() {
		return fmt.Errorf("%s is not a directory", dest)
	}
	probe := filepath.Join(dest, ".probe")
	if err := os.WriteFile(probe, nil, 0o644); err != nil {
		return err
	}
	return os.Remove(probe)
}
// call ensureWritableCleanDir(dest) before Materialize

if err := artifact.Materialize(ctx, endpoint, deps, rt, dest); err != nil {
	if strings.Contains(err.Error(), "failed to delete") {
		// permission/ownership issue at dest: attempt cleanup or fail with guidance
		return fmt.Errorf("cannot clean dest %s: %w", dest, err)
	}
	return err
}

When it happens

Trigger: Retrieving an artifact into dest when the target file exists but cannot be deleted: the file is a directory at that path, the file is immutable (chattr +i), the filesystem is read-only, or the process lacks write permission on the parent directory.

Common situations: dest path collides with a directory of the same name from a previous run with different staging names; running the worker as a non-root user against files created by root or another container layer; a mounted read-only volume as dest; Docker/Kubernetes volume permission mismatch.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• chunk write failed
• failed to both validate
• failed to flush chunks for
• failed to stat
• bad SHA256 for : , want

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4dda854d48cdbb79. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/artifact/materialize.go:220

type artifact struct {
	client         jobpb.ArtifactRetrievalServiceClient
	dep            *pipepb.ArtifactInformation
	expectedSha256 string
}

func (a artifact) retrieve(ctx context.Context, dest string) error {
	path, err := extractStagingToPath(a.dep)
	if err != nil {
		return err
	}

	filename := filepath.Join(dest, filepath.FromSlash(path))

	_, err = os.Stat(filename)
	if err == nil {
		if err = os.Remove(filename); err != nil {
			return errors.Errorf("failed to delete: %v (remove: %v)", filename, err)
		}
	} else if !os.IsNotExist(err) {
		return errors.Wrapf(err, "failed to stat %v", filename)
	}

	if err := os.MkdirAll(filepath.Dir(filename), os.ModePerm); err != nil {
		return err
	}

	stream, err := a.client.GetArtifact(ctx, &jobpb.GetArtifactRequest{Artifact: a.dep})
	if err != nil {
		return err
	}

	fd, err := os.OpenFile(filename, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0755)
	if err != nil {
		return err
	}

View on GitHub (pinned to 12126d8942)