apache/beam · error

bucket must not be empty

Error message

bucket must not be empty

What it means

parseURI in the Beam S3 filesystem adapter splits an s3:// URI into bucket and key. It throws this error when the URI has the s3 scheme but an empty host, i.e. no bucket component. This guards List, OpenRead, OpenWrite, Size, LastModified and Remove from issuing S3 API calls with an invalid bucket.

Source

Thrown at sdks/go/pkg/beam/io/filesystem/s3/util.go:37

	"errors"
	"fmt"
	"net/url"
)

// parseURI deconstructs the S3 uri in the format 's3://bucket/key' to (bucket, key)
func parseURI(uri string) (string, string, error) {
	parsed, err := url.Parse(uri)
	if err != nil {
		return "", "", err
	}

	if parsed.Scheme != "s3" {
		return "", "", errors.New("scheme must be 's3'")
	}

	bucket := parsed.Host
	if bucket == "" {
		return "", "", errors.New("bucket must not be empty")
	}

	var key string
	if parsed.Path != "" {
		key = parsed.Path[1:]
	}

	return bucket, key, nil
}

// makeURI constructs an S3 uri from the bucket and key to the format 's3://bucket/key'
func makeURI(bucket string, key string) string {
	return fmt.Sprintf("s3://%s/%s", bucket, key)
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Inspect the s3:// path being passed and ensure it includes a non-empty bucket: s3://my-bucket/key.
2. If the bucket comes from a pipeline option or template parameter, verify it is set and not an empty/unexpanded ${VAR} placeholder.
3. Strip accidental double slashes when building URIs: use url.JoinPath or fmt.Sprintf("s3://%s/%s", bucket, key) with a trimmed key.

Example fix

// before
path := fmt.Sprintf("s3:///%s", key) // bucket empty
// after
path := fmt.Sprintf("s3://%s/%s", bucket, strings.TrimPrefix(key, "/"))

u, err := url.Parse(path)
if err != nil || u.Scheme != "s3" || u.Host == "" {
    return fmt.Errorf("invalid s3 path %q: bucket must not be empty", path)
}

func isValidS3URI(path string) bool {
    u, err := url.Parse(path)
    return err == nil && u.Scheme == "s3" && u.Host != ""
}

if _, _, err := parseURI(path); err != nil {
    return fmt.Errorf("s3 filesystem: %w", err)
}

When it happens

Trigger: Calling any s3 filesystem operation (e.g. filesystem.List(ctx, 's3:///path/to/file'), OpenRead, OpenWrite, Size, LastModified, Remove) with a URI like "s3:///key" or "s3://" where the host portion between 's3://' and the next '/' is empty.

Common situations: Programmatically assembled URIs where a bucket variable was empty or unexpanded (missing pipeline option / template parameter); strings built with fmt.Sprintf("s3:///%s", key) using an extra slash; config placeholders like ${BUCKET} left unresolved in Beam pipeline options.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• empty chunk
• scheme must be 's3'
• error encoding bool: %v
• error decoding bool: %v
• error parsing S3 uri: %v

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/73c9473c337f3b9b. Report an issue: GitHub.