apache/beam · error

error getting object

Error message

error getting object %s: %v

What it means

OpenRead wraps a failed s3.GetObject call with this message. The URI parsed fine, but retrieving the object body failed — most often the object doesn't exist (NoSuchKey), the bucket is missing (NoSuchBucket), or access is denied. The filename and underlying AWS error are embedded in the message.

Solutions

1. Verify the object exists (aws s3 ls s3://bucket/key or HeadObject) before/when reading.
2. Check that the key spelling and case exactly match the stored object.
3. Grant s3:GetObject on the object prefix to the caller's IAM identity.
4. Handle smithy http.StatusNotFound / NoSuchKey in the wrapped error to produce a clearer message.

Example fix

// before
r, err := fs.OpenRead(ctx, "s3://b/key")
// after
var ae smithy.APIError
r, err := fs.OpenRead(ctx, "s3://b/key")
if errors.As(err, &ae) && ae.ErrorCode() == "NotFound" {
  return fmt.Errorf("object s3://b/key does not exist: %w", err)
}

// existence check before read
if _, err := f.client.HeadObject(ctx, &s3.HeadObjectInput{Bucket: aws.String(b), Key: aws.String(k)}); err != nil {
  return fmt.Errorf("object s3://%s/%s not readable: %w", b, k, err)
}

r, err := fs.OpenRead(ctx, uri)
if err != nil {
  var ae smithy.APIError
  if errors.As(err, &ae) && (ae.ErrorCode() == "NotFound" || ae.ErrorCode() == "NoSuchKey") {
    return fmt.Errorf("missing object %s", uri) // handle explicitly
  }
  return fmt.Errorf("read %s failed: %w", uri, err)
}

When it happens

Trigger: Calling OpenRead on s3://bucket/key where the key does not exist, the bucket doesn't exist or is in the wrong region, credentials lack s3:GetObject, or the request fails at the network level.

Common situations: Upstream job wrote to a different key than expected; case-sensitivity mismatch in key names; stale assumed-role credentials; reading a file deleted by another pipeline stage.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• error copying object
• error deleting object
• error getting metadata for object
• error getting metadata for object
• error listing object keys

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/05d19858388890f5. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/io/filesystem/s3/s3.go:131

	return objects, nil
}

// OpenRead returns a new io.ReadCloser to read contents from the file. The caller must call Close
// on the returned io.ReadCloser when done reading.
func (f *fs) OpenRead(ctx context.Context, filename string) (io.ReadCloser, error) {
	bucket, key, err := parseURI(filename)
	if err != nil {
		return nil, fmt.Errorf("error parsing S3 uri %s: %v", filename, err)
	}

	params := &s3.GetObjectInput{
		Bucket: aws.String(bucket),
		Key:    aws.String(key),
	}
	output, err := f.client.GetObject(ctx, params)
	if err != nil {
		return nil, fmt.Errorf("error getting object %s: %v", filename, err)
	}

	return output.Body, nil
}

// OpenWrite returns a new io.WriteCloser to write contents to the file. The caller must call Close
// on the returned io.WriteCloser when done writing.
func (f *fs) OpenWrite(ctx context.Context, filename string) (io.WriteCloser, error) {
	bucket, key, err := parseURI(filename)
	if err != nil {
		return nil, fmt.Errorf("error parsing S3 uri %s: %v", filename, err)
	}

	return newWriter(ctx, f.client, bucket, key), nil
}

// Size returns the size of the file.
func (f *fs) Size(ctx context.Context, filename string) (int64, error) {

View on GitHub (pinned to 12126d8942)