What causes "failed to read from %v" in apache/beam?

Verify the blob exists in GCS at the path shown in the error message (gsutil ls).

failed to read from

Error message

failed to read from %v

What it means

GetArtifact reads the staged artifact from GCS in a loop; when the underlying GCS reader returns a non-EOF error this error wraps it with the blob path. It indicates the artifact blob could not be read from its GCS location (object missing, permissions, or transient GCS failure).

Solutions

Verify the blob exists in GCS at the path shown in the error message (gsutil ls).
Check the server's GCS credentials have read access (storage.objects.get) to the bucket.
Check the bucket has no lifecycle/delete rules removing staged blobs before job submission completes.
Retry on transient errors: GCS reads occasionally fail with retryable 5xx/429 errors.
Re-stage the pipeline if the manifest references an object that no longer exists.

Defensive patterns

Strategy: retry

Validate before calling

// Verify blob readability before/at failure time (server side)
attrs, err := gcsClient.Bucket(bucket).Object(obj).Attrs(ctx)
// err != nil means object missing or inaccessible — fail fast with that cause

Try / catch

if err := getArtifact(ctx, name); err != nil {
	if strings.Contains(err.Error(), "failed to read from") {
		// re-check blob existence/permissions, then retry with backoff
	}
}

Prevention

Don't apply GCS lifecycle rules to the staging bucket that expire blobs quickly.
Ensure retrieval server credentials can read the staging bucket.
Use distinct buckets per job to avoid cross-job cleanup deleting staged blobs.
Add retry with backoff for transient GCS read errors.

When it happens

Trigger: r.Read(data) in GetArtifact returns an error that is not io.EOF; the reader was opened for the blob URI recorded in the ProxyManifest for the requested artifact name.

Common situations: The blob was deleted from the GCS bucket after staging (bucket lifecycle rules, manual cleanup); the retrieval server's credentials lack storage.objects.get on the bucket; transient GCS 5xx errors; a typo'd or stale manifest pointing at a moved object.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a7d57de5516e7c7c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/artifact/gcsproxy/retrieval.go:116

	r, err := client.Bucket(bucket).Object(object).NewReader(ctx)
	if err != nil {
		return errors.Wrapf(err, "Failed to read object for %v", key)
	}
	defer r.Close()

	data := make([]byte, 1<<20)
	for {
		n, err := r.Read(data)
		if n > 0 {
			if err := stream.Send(&jobpb.ArtifactChunk{Data: data[:n]}); err != nil {
				return errors.Wrap(err, "chunk send failed")
			}
		}
		if err == io.EOF {
			break
		}
		if err != nil {
			return errors.Wrapf(err, "failed to read from %v", blob)
		}
	}
	return nil
}

func validate(md *jobpb.ProxyManifest) error {
	keys := make(map[string]bool)
	for _, a := range md.GetManifest().GetArtifact() {
		if _, seen := keys[a.Name]; seen {
			return errors.Errorf("multiple artifact with name %v", a.Name)
		}
		keys[a.Name] = true
	}
	for _, l := range md.GetLocation() {
		fresh, seen := keys[l.Name]
		if !seen {
			return errors.Errorf("no artifact named %v for location %v", l.Name, l.Uri)
		}

View on GitHub (pinned to 12126d8942)