apache/beam · error

unable to open file

Error message

unable to open file %v

What it means

stageFile opens a local artifact file to stream it to the Beam job server during artifact staging in the universal runner. When os.Open fails, the error is wrapped as 'unable to open file %v'. This is the first step of uploading pipeline artifacts over the reverse artifact retrieval gRPC stream.

Solutions

1. Verify the artifact path exists and is readable from the launching process (os.Stat before launch).
2. Use absolute paths for artifacts so the launcher does not depend on its working directory.
3. Check file permissions and that the path is not a directory.
4. Restore the missing artifact or correct the staging configuration and relaunch.

Example fix

// before
fd, err := os.Open(filename)
if err != nil {
	return errors.Wrapf(err, "unable to open file %v", filename)
}
// after
if info, statErr := os.Stat(filename); statErr != nil || info.IsDir() {
	return errors.Errorf("artifact %q does not exist or is not a regular file", filename)
}
fd, err := os.Open(filename)

if info, err := os.Stat(artifactPath); err != nil || info.IsDir() {
	return fmt.Errorf("artifact %q missing or not a regular file", artifactPath)
}

func fileReadable(path string) bool {
	f, err := os.Open(path)
	if err != nil {
		return false
	}
	f.Close()
	return true
}

When it happens

Trigger: runnerlib.stageFile (called via stageFiles) is invoked with a path that does not exist, is a directory, lacks read permission, or resides on an unavailable volume, so os.Open returns an error.

Common situations: Launching a pipeline to a remote runner (Flink/Spark/Dataflow endpoint) with a stale, relative, or deleted artifact path; the launcher process runs in a container that does not have the artifact mounted.

Understand the failure class

Background: "open() failed", "failed to open file", "cannot create file" — what a file open error means and how to fix it — this error's family across 42 libraries.

Related errors

• chunk send failed
• failed to close stream for
• failed to scan for artifacts to stage
• failed to send chunks for
• unexpected SHA256 for sent chunks for

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e659b4f3138f0717. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/runners/universal/runnerlib/stage.go:146

						continue // so we can get the real error from stream.Recv.
					}
					return errors.Wrapf(err, "failed to stage file %v", typePl.GetPath())

				}
			default:
				return errors.Errorf("request has unexpected artifact type %s", typeUrn)
			}

		default:
			return errors.Errorf("request has unexpected type %T", request)
		}
	}
}

func stageFile(filename string, stream jobpb.ArtifactStagingService_ReverseArtifactRetrievalServiceClient) error {
	fd, err := os.Open(filename)
	if err != nil {
		return errors.Wrapf(err, "unable to open file %v", filename)
	}
	defer fd.Close()

	data := make([]byte, 1<<20)
	for {
		n, err := fd.Read(data)
		if n > 0 {
			sendErr := stream.Send(&jobpb.ArtifactResponseWrapper{
				Response: &jobpb.ArtifactResponseWrapper_GetArtifactResponse{
					GetArtifactResponse: &jobpb.GetArtifactResponse{
						Data: data[:n],
					},
				}})
			if sendErr == io.EOF {
				return sendErr
			}

			if sendErr != nil {

View on GitHub (pinned to 12126d8942)