argoproj/argo-workflows · error

failed to create new OSS client: %w

Error message

failed to create new OSS client: %w

What it means

newOSSClient wraps any failure from the aliyun-oss-go-sdk credential/client constructors with 'failed to create new OSS client'. This instance fires on the SDK-credentials path: credentials.NewCredential(nil) failed to assemble a default provider-chain credential (env vars, RAM role, OIDC-injected config). The workflow's OSS access cannot even be initialized.

Source

Thrown at workflow/artifacts/oss/oss.go:114

	// ref: https://help.aliyun.com/zh/cli/use-an-http-proxy-server#section-5yf-ejl-jwf
	if proxy, ok := os.LookupEnv("https_proxy"); ok {
		options = append(options, oss.Proxy(proxy))
	}

	if token := ossDriver.SecurityToken; token != "" {
		options = append(options, oss.SecurityToken(token))
	}

	logger := logging.RequireLoggerFromContext(ctx)
	if ossDriver.UseSDKCreds {
		// using default provider chains in sdk to get credential
		logger.Info(ctx, "Using default sdk provider chains for OSS driver")
		// need install ack-pod-identity-webhook in your cluster when using oidc provider for OSS drirver
		// the mutating webhook will help to inject the required OIDC env variables and toke volume mount configuration
		// please refer to https://www.alibabacloud.com/help/en/ack/product-overview/ack-pod-identity-webhook
		cred, err := credentials.NewCredential(nil)
		if err != nil {
			return nil, fmt.Errorf("failed to create new OSS client: %w", err)
		}
		provider := &ossCredentialsProvider{cred: cred, logger: logger}
		return oss.New(ossDriver.Endpoint, "", "", oss.SetCredentialsProvider(provider))
	}
	logger.Info(ctx, "Using AK provider")
	client, err := oss.New(ossDriver.Endpoint, ossDriver.AccessKey, ossDriver.SecretKey, options...)
	if err != nil {
		return nil, fmt.Errorf("failed to create new OSS client: %w", err)
	}
	return client, err
}

// Load downloads artifacts from OSS compliant storage, e.g., downloading an artifact into local path
func (ossDriver *ArtifactDriver) Load(ctx context.Context, inputArtifact *wfv1.Artifact, path string) error {
	err := waitutil.Backoff(defaultRetry,
		func() (bool, error) {
			logging.RequireLoggerFromContext(ctx).WithFields(logging.Fields{"path": path, "key": inputArtifact.OSS.Key}).Info(ctx, "OSS Load")
			osscli, err := ossDriver.newOSSClient(ctx)

View on GitHub (pinned to 35bff19146)

Solutions

1. Install/configure the ack-pod-identity-webhook so the pod receives OIDC credential env vars and token volume mounts.
2. Set valid OSS credential env vars (AccessKey ID/Secret, or STS token) in the container so the default provider chain resolves.
3. Fall back to static accessKey/secretKey (and optional securityToken) on the artifact's oss config instead of useSDKCreds.
4. Confirm the pod's service account has an RRSA/OIDC role bound in ACK; check with `kubectl exec ... env | grep ALIBABA`.
5. Check logs for 'Using default sdk provider chains for OSS driver' to confirm which branch failed.

Example fix

// before
- name: app
  artifacts:
    outputs:
      - name: out
        oss:
          endpoint: http://oss-cn-hangzhou.aliyuncs.com
          bucket: b
          key: k
          useSDKCreds: true  # no provider chain in pod
// after: either install ack-pod-identity-webhook, or use explicit creds
        oss:
          endpoint: http://oss-cn-hangzhou.aliyuncs.com
          bucket: b
          key: k
          accessKeySecret:
            name: oss-creds
            key: accessKey
          secretKeySecret:
            name: oss-creds
            key: secretKey

// before submitting, ensure ambient creds exist when useSDKCreds is set
required := []string{"ALIBABA_CREDENTIAL_URI", "ACCESS_KEY_ID", "OSS_ACCESS_KEY_ID"}
for _, k := range required {
	if os.Getenv(k) != "" {
		return nil
	}
}
return fmt.Errorf("no OSS ambient credentials in pod; install ack-pod-identity-webhook or set static keys")

cli, err := ossDriver.newOSSClient(ctx)
if err != nil && strings.Contains(err.Error(), "failed to create new OSS client") {
	// retry with static accessKey/secretKey fallback configuration
}

When it happens

Trigger: Using `useSDKCreds: true` on an OSS artifact without valid ambient credentials: no OSS_* env vars, no RAM/STSRoleArn, and no OIDC webhook-injected ALIBABA_CREDENTIAL_URI / token file in the pod; credentials.NewCredential(nil) returns an error.

Common situations: Forgetting to install the ack-pod-identity-webhook so OIDC env vars and token volume mounts never reach the pod; running outside ACK/RAM where the default provider chain can't resolve anything; SDK provider-chain changes in newer aliyun SDK versions.

Related errors

• failed to load system cert pool: %w
• failed to append certificates from PEM string
• not implemented
• issuer empty
• clientID empty

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/29a13d583918bbe8. Report an issue: GitHub.