argoproj/argo-workflows · error

GCS storage.NewClient: %w

Error message

GCS storage.NewClient: %w

What it means

newGCSClientDefault builds a storage.Client using Application Default Credentials (metadata server / GOOGLE_APPLICATION_CREDENTIALS). This error wraps any failure from that constructor, meaning the driver could not even create the client — almost always an ADC discovery problem in the environment, since no explicit credentials are passed.

Source

Thrown at workflow/artifacts/gcs/gcs.go:98

	return newGCSClientDefault(ctx)
}

func newGCSClientWithCredential(ctx context.Context, serviceAccountJSON string) (*storage.Client, error) {
	creds, err := google.CredentialsFromJSONWithType(ctx, []byte(serviceAccountJSON), google.ServiceAccount, storage.ScopeReadWrite)
	if err != nil {
		return nil, fmt.Errorf("GCS client CredentialsFromJSONWithType: %w", err)
	}
	client, err := storage.NewClient(ctx, option.WithCredentials(creds))
	if err != nil {
		return nil, fmt.Errorf("GCS storage.NewClient with credential: %w", err)
	}
	return client, nil
}

func newGCSClientDefault(ctx context.Context) (*storage.Client, error) {
	client, err := storage.NewClient(ctx)
	if err != nil {
		return nil, fmt.Errorf("GCS storage.NewClient: %w", err)
	}
	return client, nil
}

// Load function downloads objects from GCS
func (h *ArtifactDriver) Load(ctx context.Context, inputArtifact *wfv1.Artifact, path string) error {
	err := waitutil.Backoff(defaultRetry,
		func() (bool, error) {
			key := filepath.Clean(inputArtifact.GCS.Key)
			logger := logging.RequireLoggerFromContext(ctx)
			logger.WithFields(logging.Fields{"path": path, "key": key}).Info(ctx, "GCS Load")
			gcsClient, err := h.newGCSClient(ctx)
			if err != nil {
				logger.WithError(err).Warn(ctx, "Failed to create new GCS client")
				return !isTransientGCSErr(ctx, err), err
			}
			defer gcsClient.Close()
			err = downloadObjects(ctx, gcsClient, inputArtifact.GCS.Bucket, key, path)

View on GitHub (pinned to 35bff19146)

Solutions

1. Attach serviceAccountKeySecret with a valid service-account key to the artifact
2. Configure GKE Workload Identity: enable on node pool, bind the KSA to a GSA (annotation iam.gke.io/gcp-service-account + roles/storage.objectAdmin)
3. If relying on ADC, set GOOGLE_APPLICATION_CREDENTIALS to a mounted key file
4. Verify network access to metadata.google.internal from workflow pods

Example fix

// before: artifact with no credentials on a non-GKE cluster
artifacts: [{name: out, path: /out, gcs: {bucket: my-bucket, key: dir}}]
// after
artifacts: [{name: out, path: /out, gcs: {bucket: my-bucket, key: dir, serviceAccountKeySecret: {name: gcs-creds, key: serviceAccountKey}}}]
// kubectl create secret generic gcs-creds --from-file=serviceAccountKey=key.json

// before running GCS artifact workflows, verify ADC works in the same pod identity:
// kubectl run test-adc --serviceaccount=<ksa> --image=gcr.io/google.com/cloudsdktool/cloud-sdk \
//   -- gcloud storage ls gs://my-bucket
// success implies workload identity/ADC is wired correctly

client, err := storage.NewClient(ctx)
if err != nil {
	// ADC unavailable: fall back to explicit key
	if keyJSON := os.Getenv("GCS_SA_KEY"); keyJSON != "" {
		return newGCSClientWithCredential(ctx, keyJSON)
	}
	return err
}

When it happens

Trigger: Any GCS artifact operation when neither workload identity nor GOOGLE_APPLICATION_CREDENTIALS is available: the GCE/GKE metadata server is unreachable (non-GCP node), workload identity is not enabled on the node pool/namespace, or the ADC env var points at a missing file.

Common situations: Running Argo on non-GCP clusters (k3d/EKS) without serviceAccountKeySecret on the artifact; GKE Autopilot namespace without the Workload Identity binding (iam.gke.io/gcp-service-account annotation); metadata server blocked by NetworkPolicy.

Related errors

• GCS client CredentialsFromJSONWithType: %w
• GCS storage.NewClient with credential: %w
• --client-certificate and --client-key must be provided toget
• AuthSupplier cannot be empty when connecting to Argo Server
• must specify at least one auth mode

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/ee68da56631f0f87. Report an issue: GitHub.