apache/beam · error

error opening destination file

Error message

error opening destination file %s: %w

What it means

During extraction of an Uber JAR (extractJar, called from MakeJar), a file inside the JAR could not be written to the destination directory. os.OpenFile with O_WRONLY|O_CREATE|O_TRUNC failed, typically because the cache directory tree is not writable or a same-named path is a directory. The library wraps the OS error with the destination file name so the failing entry is identifiable.

Solutions

1. Check that the directory holding the JAR cache (under $HOME/.beam661bea09-.../tmpDir) exists and is writable by the running user (ls -ld, chown/chmod).
2. Free disk space (df -h) and retry; a full filesystem makes O_CREATE fail with ENOSPC.
3. If an entry-name/directory collision is suspected, delete the stale tmpDir cache directory and re-run so extraction starts clean.
4. Verify the JAR is not corrupt (unzip -t mainJar) and that entry names are sane.

Example fix

// before: cache dir owned by root in container
USER root
// after: run the expansion service as a user that owns its home
RUN mkdir -p /home/beam && chown beam:beam /home/beam
USER beam

destDir := filepath.Dir(fileName)
if info, err := os.Stat(destDir); err != nil || !info.IsDir() {
    return fmt.Errorf("dest dir %s missing: %w", destDir, err)
}
if fi, err := os.Stat(fileName); err == nil && fi.IsDir() {
    return fmt.Errorf("%s exists as a directory; remove collision", fileName)
}
if usable, err := hasFreeSpace(destDir); err == nil && !usable { /* abort */ }

df, err := os.OpenFile(fileName, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0777)
if err != nil {
    var pe *fs.PathError
    if errors.As(err, &pe) && errors.Is(pe.Err, fs.ErrPermission) {
        // fix perms / re-run as correct user
    }
    return fmt.Errorf("error opening destination file %s: %w", fileName, err)
}

When it happens

Trigger: extractJar(mainJar, tmpDir) walks entries of the source JAR and calls os.OpenFile(fileName, O_WRONLY|O_CREATE|O_TRUNC, 0777) for each non-directory entry; failure occurs when the dest dir lacks write permission, disk is full, a JAR entry name collides with an existing directory on disk, or the path is invalid on the OS.

Common situations: Read-only $HOME or full disk when the jar cache (~/.beam661bea09-5dd8-4b9a-ac72-270a895bd3b1 by default) is written; corrupted or adversarially crafted JAR whose entry names collide with directories created earlier in the same extraction; running the expansion service as a non-root container user with an unwritable home.

Understand the failure class

Background: "open() failed", "failed to open file", "cannot create file" — what a file open error means and how to fix it — this error's family across 42 libraries.

Related errors

• chunk write failed
• error copying file
• error creating jar packJar
• error creating jarFile header
• error in creating relative path

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ae782b72e556cbf7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/core/runtime/xlangx/expansionx/download.go:182

	for _, file := range reader.File {
		fileName, err := validatePath(dest, file.Name)
		if err != nil {
			return fmt.Errorf("error validating file path (%s, %s): %w", dest, file.Name, err)
		}
		if file.FileInfo().IsDir() {
			os.MkdirAll(fileName, 0700)
			continue
		}

		sf, err := file.Open()
		if err != nil {
			return fmt.Errorf("error opening source file %s: %w", file.Name, err)
		}
		defer sf.Close()

		df, err := os.OpenFile(fileName, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0777)
		if err != nil {
			return fmt.Errorf("error opening destination file %s: %w", fileName, err)
		}
		defer df.Close()

		if _, err := io.Copy(df, sf); err != nil {
			return err
		}
	}
	return nil
}

func packJar(source, dest string) error {
	jar, err := os.Create(dest)
	if err != nil {
		return fmt.Errorf("error creating jar packJar(%s,%s)=%w", source, dest, err)
	}
	defer jar.Close()

	jarFile := zip.NewWriter(jar)

View on GitHub (pinned to 12126d8942)