apache/beam · error

object must have bucket

Error message

object %s must have bucket

What it means

gcsx.ParseObject requires the parsed gs:// URL to have a bucket host. This error is returned when the scheme is 'gs' but the URL has no host component — e.g. the bare string 'gs://' or 'gs:///path' with no bucket name.

Solutions

1. Include the bucket in the URI: 'gs://bucket-name/object-path'
2. Check the config/env providing the bucket name isn't empty
3. Validate with a quick parse before calling: u, _ := url.Parse(obj); ensure u.Scheme=="gs" && u.Host!=""

Example fix

// before
gcsx.OpenRead(ctx, client, "gs://" + bucket + "/obj") // bucket == ""
// after
if bucket == "" { return errors.New("bucket not configured") }
gcsx.OpenRead(ctx, client, "gs://" + bucket + "/obj")

u, err := url.Parse(obj)
if err == nil && u.Scheme == "gs" && u.Host == "" {
	return fmt.Errorf("gs URI %q missing bucket", obj)
}

func hasGCSBucket(s string) bool {
	u, err := url.Parse(s)
	return err == nil && u.Scheme == "gs" && u.Host != ""
}

if err != nil {
	if strings.Contains(err.Error(), "must have bucket") {
		return fmt.Errorf("GCS URI %q missing bucket name", obj)
	}
	return err
}

When it happens

Trigger: Calling gcsx.ParseObject (directly or via List, OpenRead, staging/retrieval servers, ReadProxyManifest) with 'gs://' or 'gs:///object' where parsed.Host is empty.

Common situations: Configuration with a truncated GCS URI, template variables left unexpanded (e.g. 'gs://%s/' with an empty variable), or staging paths built from an empty bucket-name config value.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• object must have 'gs' scheme
• invalid manifest location
• location is not a GCS object
• project cannot be empty, got
• A VPC network must be provided to use a private endpoint.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a2c19848397f8f2b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/util/gcsx/gcs.go:145

func MakeObject(bucket, path string) string {
	if bucket == "" {
		panic("bucket must be non-empty")
	}
	return fmt.Sprintf("gs://%v/%v", bucket, path)
}

// ParseObject deconstructs a GCS object name into (bucket, name).
func ParseObject(object string) (bucket, path string, err error) {
	parsed, err := url.Parse(object)
	if err != nil {
		return "", "", err
	}

	if parsed.Scheme != "gs" {
		return "", "", errors.Errorf("object %s must have 'gs' scheme", object)
	}
	if parsed.Host == "" {
		return "", "", errors.Errorf("object %s must have bucket", object)
	}
	if parsed.Path == "" {
		return parsed.Host, "", nil
	}

	// remove leading "/" in URL path
	return parsed.Host, parsed.Path[1:], nil
}

// Join joins a GCS path with an element. Preserves
// the gs:// prefix.
func Join(object string, elms ...string) string {
	bucket, prefix, err := ParseObject(object)
	if err != nil {
		panic(err)
	}
	return MakeObject(bucket, path.Join(prefix, path.Join(elms...)))
}

View on GitHub (pinned to 12126d8942)