apache/beam · error

invalid manifest object

Error message

invalid manifest object %v

What it means

ReadProxyManifest parses a GCS proxy manifest object path before reading it. This wrapped error is returned when gcsx.ParseObject cannot interpret the object string as a valid GCS object reference (bucket/object), so the manifest location itself is malformed.

Solutions

1. Pass the full GCS object path in gs://bucket/object form, e.g. gs://my-bucket/artifacts/manifest.
2. Verify no surrounding whitespace or truncation of the path in your config/env.
3. Use gcsx.ParseObject yourself first to validate the string and surface a clearer failure.

Example fix

// before
ReadProxyManifest(ctx, "my-bucket/artifacts/manifest")
// after
ReadProxyManifest(ctx, "gs://my-bucket/artifacts/manifest")

func isGCSPath(s string) bool { _, _, err := gcsx.ParseObject(s); return err == nil }
if !isGCSPath(manifestObj) { return fmt.Errorf("manifest must be gs://bucket/object, got %q", manifestObj) }

func isGCSObject(s string) bool { return strings.HasPrefix(s, "gs://") && strings.Count(strings.TrimPrefix(s, "gs://"), "/") >= 1 }

md, err := ReadProxyManifest(ctx, obj); if err != nil { if strings.Contains(err.Error(), "invalid manifest object") { /* fix path format */ } ; return err }

When it happens

Trigger: Calling ReadProxyManifest(ctx, object) with a string that is not of the form gs://bucket/path, e.g. an empty string, a bare bucket name, or an https URL.

Common situations: Constructing the manifest path by hand, missing the gs:// scheme, or accidentally passing the manifest contents/URL instead of the GCS object path.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

• location is not a GCS object
• panic(err)
• bad coder kind
• bad CoGBK
• bad combinefn

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0fdd2e84f59b990a. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/artifact/gcsproxy/retrieval.go:41

	jobpb "github.com/apache/beam/sdks/v2/go/pkg/beam/model/jobmanagement_v1"
	"github.com/apache/beam/sdks/v2/go/pkg/beam/util/gcsx"
	"golang.org/x/net/context"
	"google.golang.org/protobuf/proto"
)

// RetrievalServer is a artifact retrieval server backed by Google
// Cloud Storage (GCS). It serves a single manifest and ignores
// the worker id. The server performs no caching or pre-fetching.
type RetrievalServer struct {
	md    *jobpb.Manifest
	blobs map[string]string
}

// ReadProxyManifest reads and parses the proxy manifest from GCS.
func ReadProxyManifest(ctx context.Context, object string) (*jobpb.ProxyManifest, error) {
	bucket, obj, err := gcsx.ParseObject(object)
	if err != nil {
		return nil, errors.Wrapf(err, "invalid manifest object %v", object)
	}

	cl, err := gcsx.NewClient(ctx, storage.ScopeReadOnly)
	if err != nil {
		return nil, errors.Wrap(err, "failed to create GCS client")
	}
	content, err := gcsx.ReadObject(ctx, cl, bucket, obj)
	if err != nil {
		return nil, errors.Wrapf(err, "failed to read manifest %v", object)
	}
	var md jobpb.ProxyManifest
	if err := proto.Unmarshal(content, &md); err != nil {
		return nil, errors.Wrapf(err, "invalid manifest %v", object)
	}
	return &md, nil
}

// NewRetrievalServer creates a artifact retrieval server for the

View on GitHub (pinned to 12126d8942)