argoproj/argo-workflows · error

failed to get service account secret: %w

Error message

failed to get service account secret: %w

What it means

After a rule-matched service account is chosen, the server builds a per-user k8s client by reading the SA's token Secret (secrets.TokenNameForServiceAccount). This error wraps a failure fetching that secret from the informer cache — typically the secret doesn't exist or isn't readable, so the server cannot impersonate the matched service account. On Kubernetes 1.24+ long-lived SA token secrets are no longer auto-created, so this frequently fires when relying on legacy auto-generated token secrets.

Source

Thrown at server/auth/gatekeeper.go:326

	fields := logging.Fields{
		"serviceAccount":       delegatedAccount.Name,
		"subject":              claims.Subject,
		"email":                claims.Email,
		"ssoDelegationAllowed": ssoDelegationAllowed,
		"ssoDelegated":         ssoDelegated,
	}
	if loginAccount != nil {
		fields["loginServiceAccount"] = loginAccount.Name
	}
	logger.WithFields(fields).Info(ctx, "selected SSO RBAC service account for user")
	return s.getClientsForServiceAccount(ctx, claims, delegatedAccount)
}

func (s *gatekeeper) authorizationForServiceAccount(ctx context.Context, serviceAccount *corev1.ServiceAccount) (string, error) {
	secretName := secrets.TokenNameForServiceAccount(serviceAccount)
	secret, err := s.cache.GetSecret(ctx, serviceAccount.GetNamespace(), secretName)
	if err != nil {
		return "", fmt.Errorf("failed to get service account secret: %w", err)
	}
	return "Bearer " + string(secret.Data["token"]), nil
}

func addClaimsLogFields(claims *authTypes.Claims, fields logging.Fields) logging.Fields {
	if fields == nil {
		fields = logging.Fields{}
	}
	fields["subject"] = claims.Subject
	if claims.Email != "" {
		fields["email"] = claims.Email
	}
	return fields
}

func DefaultClientForAuthorization(authorization string, config *rest.Config) (*rest.Config, *servertypes.Clients, error) {
	restConfig, err := kubeconfig.GetRestConfig(authorization)
	if err != nil {

View on GitHub (pinned to 35bff19146)

Solutions

1. Create a long-lived token secret properly and reference it: a secret of type kubernetes.io/service-account-token with annotation kubernetes.io/service-account.name: <sa>; k8s will fill the token field.
2. On K8s >=1.24, prefer creating the secret explicitly (legacy auto-generation is gone) and restart/re-sync so the informer cache sees it.
3. Verify the secret exists: `kubectl -n <ns> get secret` for the name TokenNameForServiceAccount derives (`<sa-name>-token-...` or the secret named via the SA's secrets list), and that its `token` key is populated.
4. Check argo-server's RBAC allows reading secrets in the SSO/delegated namespaces, and that the SA has the secret listed under `secrets:`.

Example fix

# before: SA with no token secret (k8s 1.24+)
kind: ServiceAccount
metadata:
  name: argo-sso
# after: explicit long-lived token secret
kind: ServiceAccount
metadata:
  name: argo-sso
secrets:
  - name: argo-sso-token
---
apiVersion: v1
kind: Secret
metadata:
  name: argo-sso-token
  annotations:
    kubernetes.io/service-account.name: argo-sso
type: kubernetes.io/service-account-token

SA=<name>; NS=<ns>
kubectl -n $NS get secret "${SA}-token" -o jsonpath='{.type} {.data.token}'  # expect kubernetes.io/service-account-token and a non-empty token

// server retries via informer cache; client sees PermissionDenied/Unauthenticated
if st, ok := status.FromError(err); ok && st.Code() == codes.PermissionDenied {
    time.Sleep(2 * time.Second) // allow secret propagation into informer cache, then retry once
}

When it happens

Trigger: SSO RBAC matched a ServiceAccount, but GetSecret(ns, <sa-name>-token-xxx) fails: the token secret was deleted, never created (K8s >= 1.24 no auto token secrets), or was created without the kubernetes.io/service-account-token type / correct sa annotation.

Common situations: Upgrading to Kubernetes 1.24+ where SA token secrets are no longer auto-generated; manually created SAs in clusters where the controller manager's token cleaner removed the secret; creating a secret with the wrong `type` or missing `kubernetes.io/service-account.name` annotation; RBAC preventing argo-server's service account from reading secrets in the target namespace.

Related errors

• failed to create secret: %w
• failed to read secret: %w
• failed to list SSO RBAC service accounts: %w
• failed to parse private key. If you have already defined a S
• failed to parse private key: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/d4c2b9f2bed5b341. Report an issue: GitHub.