JuliusBrussee/caveman · error

awscreds: read web identity token file

Error message

awscreds: read web identity token file: %w

What it means

awscreds' web identity source reads the token from the file named by AWS_WEB_IDENTITY_TOKEN_FILE (used with EKS IRSA / pod identity). If both AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN are set but os.ReadFile fails — missing file, bad path, permissions — the OS error is wrapped with this message and returned, aborting the credential chain.

Solutions

  1. Verify the file exists and is readable at the exact path in AWS_WEB_IDENTITY_TOKEN_FILE (ls/cat it from the same container)
  2. Ensure the serviceaccount token volume is mounted — in EKS check that the pod spec uses a serviceaccount with the EKS pod identity webhook, and retry after startup if the mount races
  3. Fix the env value if it points to a stale/host path
  4. If web identity is not intended, unset AWS_WEB_IDENTITY_TOKEN_FILE so the source is skipped instead of erroring

Example fix

// before
os.Setenv("AWS_WEB_IDENTITY_TOKEN_FILE", "~/.aws/token") // ~ never expands in os.ReadFile
// after
tokenFile := os.Getenv("AWS_WEB_IDENTITY_TOKEN_FILE")
if tokenFile != "" {
    if _, err := os.Stat(tokenFile); err != nil {
        tokenFile = "/var/run/secrets/eks.amazonaws.com/serviceaccount/token"
    }
    os.Setenv("AWS_WEB_IDENTITY_TOKEN_FILE", tokenFile)
}
creds, err := awscreds.Credentials(ctx, p)
Defensive patterns

Strategy: try-catch

Validate before calling

if tf := os.Getenv("AWS_WEB_IDENTITY_TOKEN_FILE"); tf != "" {
    if fi, err := os.Stat(tf); err != nil || fi.Size() == 0 {
        return fmt.Errorf("web identity token file %q missing or empty", tf)
    }
}

Try / catch

creds, err := awscreds.Credentials(ctx, p)
var pe *fs.PathError
if err != nil && strings.Contains(err.Error(), "read web identity token file") && errors.As(err, &pe) {
    log.Printf("token file unreadable (%v); falling back to env creds", pe)
    creds = envCredsFallback()
}

Prevention

When it happens

Trigger: fromWebIdentity is entered because AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN are both set, then os.ReadFile(tokenFile) returns an error: file deleted after mount, wrong path in env, volume not yet mounted, permission denied.

Common situations: Kubernetes pods where the serviceaccount token project volume isn't mounted yet (race at startup); a copied .env setting the token path to a host path that doesn't exist in the container; readdir/permission changes on /var/run/secrets/eks.amazonaws.com/serviceaccount/.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/4f952313b011f79d. Report an issue: GitHub.

Appendix: source

Thrown at shared/platform/awscreds/awscreds.go:312

			AccessKeyID:     access,
			SecretAccessKey: secret,
			SessionToken:    p.env("AWS_SESSION_TOKEN"),
		},
		source: "env",
	}, nil
}

// fromWebIdentity implements the EKS IRSA / generic OIDC flow: exchange the
// projected service account token for role credentials at STS. The call is
// unsigned by definition — the token is the proof.
func (p *Provider) fromWebIdentity(ctx context.Context) (*result, error) {
	tokenFile, roleARN := p.env("AWS_WEB_IDENTITY_TOKEN_FILE"), p.env("AWS_ROLE_ARN")
	if tokenFile == "" || roleARN == "" {
		return nil, nil
	}
	raw, err := os.ReadFile(tokenFile)
	if err != nil {
		return nil, fmt.Errorf("awscreds: read web identity token file: %w", err)
	}
	token := strings.TrimSpace(string(raw))
	if token == "" {
		return nil, errors.New("awscreds: web identity token file is empty")
	}
	sessionName := p.env("AWS_ROLE_SESSION_NAME")
	if sessionName == "" {
		sessionName = fmt.Sprintf("caveman-proxy-%d", p.now().Unix())
	}
	form := url.Values{
		"Action":           {"AssumeRoleWithWebIdentity"},
		"Version":          {"2011-06-15"},
		"RoleArn":          {roleARN},
		"RoleSessionName":  {sessionName},
		"WebIdentityToken": {token},
		"DurationSeconds":  {"3600"},
	}
	req, err := http.NewRequestWithContext(ctx, http.MethodPost, p.stsEndpoint, strings.NewReader(form.Encode()))

View on GitHub (pinned to 3ee70a1026)