apache/beam · error

Failed to create client for

Error message

Failed to create client for %v

What it means

GetArtifact creates a read-only GCS client to stream the artifact blob. This wrapped error is returned when gcsx.NewClient fails during artifact retrieval, i.e. client/auth initialization failed for the given artifact request.

Solutions

1. Attach a service account with Storage Object Viewer to the retrieval service or set GOOGLE_APPLICATION_CREDENTIALS to a valid key file.
2. Run gcloud auth application-default login in local/dev environments.
3. Verify network access to storage.googleapis.com and that Cloud Storage API is enabled.

Example fix

// before: deployment without a storage role
// after: grant roles/storage.objectViewer to the service account, then redeploy

if os.Getenv("GOOGLE_APPLICATION_CREDENTIALS") == "" {
    if _, err := gcp.DefaultCredentials(ctx); err != nil { return fmt.Errorf("GCS credentials required before artifact retrieval") }
}

err := streamArtifact(ctx, key)
if err != nil && strings.Contains(err.Error(), "Failed to create client") { return retryable(err) }

When it happens

Trigger: Calling GetArtifact when credentials are missing or invalid in the retrieval server's environment: no Application Default Credentials, bad GOOGLE_APPLICATION_CREDENTIALS, or library init failure.

Common situations: Running the retrieval service outside GCP without attached service accounts, revoked or expired key files, or Storage API disabled.

Related errors

• failed to create GCS client
• failed to read manifest
• Failed to read object for
• Artifact not found at
• Can not get unique key from solr

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fbe456652de0dea7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/go/pkg/beam/artifact/gcsproxy/retrieval.go:94

// GetManifest returns the manifest for all artifacts.
func (s *RetrievalServer) GetManifest(ctx context.Context, req *jobpb.GetManifestRequest) (*jobpb.GetManifestResponse, error) {
	return &jobpb.GetManifestResponse{Manifest: s.md}, nil
}

// GetArtifact returns a given artifact.
func (s *RetrievalServer) GetArtifact(req *jobpb.LegacyGetArtifactRequest, stream jobpb.LegacyArtifactRetrievalService_GetArtifactServer) error {
	key := req.GetName()
	blob, ok := s.blobs[key]
	if !ok {
		return errors.Errorf("artifact %v not found", key)
	}

	bucket, object := parseObject(blob)

	ctx := stream.Context()
	client, err := gcsx.NewClient(ctx, storage.ScopeReadOnly)
	if err != nil {
		return errors.Wrapf(err, "Failed to create client for %v", key)
	}

	// Stream artifact in up to 1MB chunks.
	r, err := client.Bucket(bucket).Object(object).NewReader(ctx)
	if err != nil {
		return errors.Wrapf(err, "Failed to read object for %v", key)
	}
	defer r.Close()

	data := make([]byte, 1<<20)
	for {
		n, err := r.Read(data)
		if n > 0 {
			if err := stream.Send(&jobpb.ArtifactChunk{Data: data[:n]}); err != nil {
				return errors.Wrap(err, "chunk send failed")
			}
		}
		if err == io.EOF {

View on GitHub (pinned to 12126d8942)