argoproj/argo-workflows · critical

failed to get token: %w

Error message

failed to get token: %w

What it means

After constructing an azidentity credential, azureConnector.Connect requests an access token via cred.GetToken for the configured scope (e.g. https://ossrdbms-aad.database.windows.net/.default). If the token request fails (auth endpoint error, bad credentials, network, expired federated token), Connect wraps it with "failed to get token". Unlike error 352, a credential object existed but using it failed.

Source

Thrown at util/sqldb/azure_auth.go:27

	"github.com/Azure/azure-sdk-for-go/sdk/azcore/policy"
	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	"github.com/lib/pq"
)

type azureConnector struct {
	dsn   string
	scope string
}

func (c *azureConnector) Connect(ctx context.Context) (driver.Conn, error) {
	cred, err := azidentity.NewDefaultAzureCredential(nil)
	if err != nil {
		return nil, fmt.Errorf("failed to obtain a credential: %w", err)
	}

	token, err := cred.GetToken(ctx, policy.TokenRequestOptions{Scopes: []string{c.scope}})
	if err != nil {
		return nil, fmt.Errorf("failed to get token: %w", err)
	}

	// Escape single quotes in token just in case
	escapedToken := strings.ReplaceAll(token.Token, "'", "\\'")

	// Append password to DSN
	dsnWithPassword := fmt.Sprintf("%s password='%s'", c.dsn, escapedToken)

	return pq.Driver{}.Open(dsnWithPassword)
}

func (c *azureConnector) Driver() driver.Driver {
	return pq.Driver{}
}

View on GitHub (pinned to 35bff19146)

Solutions

1. Verify the credential's identity is valid: check client secret/cert expiry, or re-sync the federated identity credential in Azure.
2. Confirm the configured scope matches the target service (for Azure Database for PostgreSQL it is https://ossrdbms-aad.database.windows.net/.default).
3. Ensure network egress to the authority endpoint (login.microsoftonline.com or the configured AZURE_AUTHORITY_HOST) from the pod.
4. Grant the managed/workload identity access to the database (CREATE USER ... FROM EXTERNAL PROVIDER) so the DB accepts the token.
5. Inspect the wrapped azidentity error for HTTP status/MSAL detail (invalid_client, unauthorized_client, network timeout).

Example fix

// before: scope mismatch
scope: "https://database.windows.net/.default"
// after
scope: "https://ossrdbms-aad.database.windows.net/.default"

if !strings.HasSuffix(scope, "/.default") {
    return fmt.Errorf("scope %q should end in /.default for DB tokens", scope)
}

var tok policy acquireErr
if err != nil {
    if strings.Contains(err.Error(), "failed to get token") {
        // transient network failures are retryable; invalid_client is not
        if strings.Contains(err.Error(), "invalid_client") {
            log.Fatal(err)
        }
        backoff.Retry(func() error { return connectAzure(ctx) }, strategy)
    }
}

When it happens

Trigger: cred.GetToken returns an error when the credential cannot authenticate against Microsoft Entra ID: invalid client secret/certificate, workload identity federated token expired or file unreadable, managed identity unavailable on the host, MSAL/authority endpoint unreachable, or scope string is invalid.

Common situations: Federated identity credential deleted or name mismatched in Azure; AAD role not granted to the Postgres user so token issuance for the DB scope is denied; corporate proxy blocking login.microsoftonline.com; AZURE_AUTHORITY_HOST pointing to a sovereign cloud while resources are elsewhere.

Related errors

• failed to obtain a credential: %w
• unable to create default Azure credential: %w
• unable to create Azure shared key credential: %w
• unable to download blob %s: %w
• --client-certificate and --client-key must be provided toget

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/a1aca1aa5703abeb. Report an issue: GitHub.