apache/beam · error

location is not a GCS object

Error message

location %v is not a GCS object

What it means

NewRetrievalServer validates every artifact location in the manifest is a GCS object by parsing each URI. This wrapped error is returned when any location URI in the ProxyManifest cannot be parsed as a GCS bucket/object pair.

Solutions

1. Correct the manifest so every location uri is a valid gs://bucket/object path.
2. Regenerate the manifest with a GCS-backed staging/proxy pipeline.
3. Pre-validate each URI with gcsx.ParseObject before constructing the server to fail fast with a clearer message.

Example fix

// before
Location: &jobpb.ArtifactMetadata{Uri: "/tmp/artifact.bin"}
// after
Location: &jobpb.ArtifactMetadata{Uri: "gs://my-bucket/artifacts/artifact.bin"}

for _, l := range md.GetLocation() {
    if _, _, err := gcsx.ParseObject(l.GetUri()); err != nil {
        return fmt.Errorf("location %q is not gs://bucket/object", l.GetUri())
    }
}

func isGCSLocation(l *jobpb.Location) bool { _, _, err := gcsx.ParseObject(l.GetUri()); return err == nil }

srv, err := NewRetrievalServer(ctx, md)
if err != nil { return fmt.Errorf("manifest has non-GCS locations: %w", err) }

When it happens

Trigger: Building a retrieval server from a manifest whose Location entries contain non-GCS URIs (local file paths, https URLs, empty strings) via NewRetrievalServer(ctx, md).

Common situations: Manifests produced by non-GCS artifact proxies, hand-edited manifests, or URIs missing the gs:// prefix.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

• invalid manifest location
• invalid manifest object
• object must have bucket
• object must have 'gs' scheme
• Compression factor should be greater than 0.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/83a5ba077589e7d2. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/artifact/gcsproxy/retrieval.go:69

	}
	var md jobpb.ProxyManifest
	if err := proto.Unmarshal(content, &md); err != nil {
		return nil, errors.Wrapf(err, "invalid manifest %v", object)
	}
	return &md, nil
}

// NewRetrievalServer creates a artifact retrieval server for the
// given manifest. It requires that the locations are in GCS.
func NewRetrievalServer(md *jobpb.ProxyManifest) (*RetrievalServer, error) {
	if err := validate(md); err != nil {
		return nil, err
	}

	blobs := make(map[string]string)
	for _, l := range md.GetLocation() {
		if _, _, err := gcsx.ParseObject(l.GetUri()); err != nil {
			return nil, errors.Wrapf(err, "location %v is not a GCS object", l.GetUri())
		}
		blobs[l.GetName()] = l.GetUri()
	}
	return &RetrievalServer{md: md.GetManifest(), blobs: blobs}, nil
}

// GetManifest returns the manifest for all artifacts.
func (s *RetrievalServer) GetManifest(ctx context.Context, req *jobpb.GetManifestRequest) (*jobpb.GetManifestResponse, error) {
	return &jobpb.GetManifestResponse{Manifest: s.md}, nil
}

// GetArtifact returns a given artifact.
func (s *RetrievalServer) GetArtifact(req *jobpb.LegacyGetArtifactRequest, stream jobpb.LegacyArtifactRetrievalService_GetArtifactServer) error {
	key := req.GetName()
	blob, ok := s.blobs[key]
	if !ok {
		return errors.Errorf("artifact %v not found", key)
	}

View on GitHub (pinned to 12126d8942)