apache/beam · error

failed to create GCS client

Error message

failed to create GCS client

What it means

CommitManifest creates a GCS client with read-write scope to upload the marshaled ProxyManifest. Failure to construct that client (missing credentials, no network to Google APIs, bad environment) is wrapped with this message.

Solutions

  1. Set GOOGLE_APPLICATION_CREDENTIALS to a service-account JSON key with storage read-write access, or run `gcloud auth application-default login`.
  2. Run on GCP infrastructure (GCE/GKE/Cloud Run) where the metadata server supplies credentials automatically.
  3. Verify network access to oauth2.googleapis.com and storage.googleapis.com (proxies/VPN/firewalls).
  4. Confirm the GOOGLE_CLOUD_PROJECT or quota project is set if required by your auth setup.

Example fix

// before
commitCtx := context.Background()
// after: ensure credentials exist first, with a clear failure
creds, err := google.FindDefaultCredentials(ctx, storage.ScopeReadWrite)
if err != nil {
	return nil, fmt.Errorf("set GOOGLE_APPLICATION_CREDENTIALS: %w", err)
}
_ = creds
// then call CommitManifest(ctx, token) with ctx carrying auth metadata
Defensive patterns

Strategy: try-catch

Validate before calling

// Detect credential availability before staging
if _, err := google.FindDefaultCredentials(ctx, storage.ScopeReadWrite); err != nil {
	return fmt.Errorf("no GCP credentials: set GOOGLE_APPLICATION_CREDENTIALS")
}

Try / catch

if err := stage(...); err != nil {
	if strings.Contains(err.Error(), "failed to create GCS client") {
		// check GOOGLE_APPLICATION_CREDENTIALS / metadata server, then retry
	}
}

Prevention

When it happens

Trigger: gcsx.NewClient(ctx, storage.ScopeReadWrite) returns an error inside CommitManifest — typically google.FindDefaultCredentials fails because no Application Default Credentials are available.

Common situations: Running outside GCP without GOOGLE_APPLICATION_CREDENTIALS set; no gcloud application-default credentials configured; metadata server unreachable; network/firewall blocking oauth2.googleapis.com; service account JSON key file path wrong.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1ddc02b96aa66e9c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/artifact/gcsproxy/staging.go:85

func (s *StagingServer) CommitManifest(ctx context.Context, req *jobpb.CommitManifestRequest) (*jobpb.CommitManifestResponse, error) {
	manifest := req.GetManifest()

	s.mu.Lock()
	loc, err := matchLocations(manifest.GetArtifact(), s.blobs)
	if err != nil {
		s.mu.Unlock()
		return nil, err
	}
	s.mu.Unlock()

	data, err := proto.Marshal(&jobpb.ProxyManifest{Manifest: manifest, Location: loc})
	if err != nil {
		return nil, errors.Wrap(err, "failed to marshal proxy manifest")
	}

	cl, err := gcsx.NewClient(ctx, storage.ScopeReadWrite)
	if err != nil {
		return nil, errors.Wrap(err, "failed to create GCS client")
	}
	if err := gcsx.WriteObject(ctx, cl, s.bucket, s.manifest, bytes.NewReader(data)); err != nil {
		return nil, errors.Wrap(err, "failed to write manifest")
	}

	// Commit returns the location of the manifest as the token, which can
	// then be used to configure the retrieval proxy. It is redundant right
	// now, but would be needed for a staging server that serves multiple
	// jobs. Such a server would also use the ID sent with each request.

	return &jobpb.CommitManifestResponse{RetrievalToken: gcsx.MakeObject(s.bucket, s.manifest)}, nil
}

// matchLocations ensures that all artifacts have been staged and have valid
// content. It is fine for staged artifacts to not appear in the manifest.
func matchLocations(artifacts []*jobpb.ArtifactMetadata, blobs map[string]staged) ([]*jobpb.ProxyManifest_Location, error) {
	var loc []*jobpb.ProxyManifest_Location
	for _, a := range artifacts {

View on GitHub (pinned to 12126d8942)