kubernetes/kops · error
failed to load AWS config: %w
Error message
failed to load AWS config: %w
What it means
getRegionFromMetadata loads an AWS SDK v2 config (to then query EC2 IMDS for the current region) with a 100ms HTTP client. If awsconfig.LoadDefaultConfig itself returns an error — typically invalid shared config/credentials files or a malformed profile — this error wraps it. It occurs while kOps tries to infer the region for a state-store bucket by asking the local instance metadata service.
Source
Thrown at util/pkg/vfs/s3context.go:349
}
klog.V(2).Infof("GOOS=%q, assuming not running on EC2", runtime.GOOS)
return false, nil
}
// getRegionFromMetadata queries the metadata service for the current region, if running in EC2
func getRegionFromMetadata(ctx context.Context) (string, error) {
ctx, span := tracer.Start(ctx, "getRegionFromMetadata")
defer span.End()
// Use an even shorter timeout, to minimize impact when not running on EC2
// Note that we still retry a few times, this works out a little under a 1s delay
shortTimeout := &http.Client{
Timeout: 100 * time.Millisecond,
}
config, err := awsconfig.LoadDefaultConfig(ctx, awsconfig.WithHTTPClient(shortTimeout))
if err != nil {
return "", fmt.Errorf("failed to load AWS config: %w", err)
}
client := imds.NewFromConfig(config)
metadataRegion, err := client.GetRegion(ctx, &imds.GetRegionInput{})
if err != nil {
return "", fmt.Errorf("getting AWS region from metadata: %w", err)
}
return metadataRegion.Region, nil
}
func VFSPath(url string) (string, error) {
if !s3UrlRegexp.MatchString(url) {
return "", fmt.Errorf("%s is not a valid S3 URL", url)
}
groupNames := s3UrlRegexp.SubexpNames()
result := s3UrlRegexp.FindAllStringSubmatch(url, -1)[0]View on GitHub (pinned to 4c8573c808)
Solutions
- Inspect the wrapped message after 'failed to load AWS config:' — it names the offending file/profile; fix the syntax or reference
- Run aws configure list / AWS_PROFILE=<profile> aws sts get-caller-identity to validate the profile resolves correctly
- Fix role_arn/source_profile entries so the caller is in the trust policy and can assume the role
- Point AWS_CONFIG_FILE / AWS_SHARED_CREDENTIALS_FILE at valid files, or unset them if stale
- If region inference keeps failing, bypass metadata inference: set AWS_REGION explicitly or store cluster state with a fully qualified s3://bucket that kOps can resolve via HeadBucket
Example fix
// before: broken ~/.aws/config [profile kops] role_arn = arn:aws:iam::123:role/Kops // missing source_profile → config load/credential resolution fails // after [profile kops] role_arn = arn:aws:iam::123:role/Kops source_profile = default
Defensive patterns
Strategy: validation
Validate before calling
// Validate the AWS config resolves before running kops
aws configure list >/dev/null 2>&1 || { echo "invalid AWS config files"; exit 1; }
AWS_PROFILE=${AWS_PROFILE:-default} aws sts get-caller-identity >/dev/null || { echo "profile cannot authenticate"; exit 1; } Type guard
func isAWSConfigLoadError(err error) bool {
return err != nil && strings.Contains(err.Error(), "failed to load AWS config")
} Prevention
- Keep ~/.aws/config and ~/.aws/credentials valid INI; validate with `aws configure list` after edits
- Ensure every role_arn has a resolvable source_profile and that the caller is in the role's trust policy
- Avoid stale AWS_CONFIG_FILE / AWS_SHARED_CREDENTIALS_FILE overrides; unset if unused
- Test `aws sts get-caller-identity` with the intended profile before running kops
- Set AWS_REGION explicitly so region inference via IMDS (this code path) is avoided
When it happens
Trigger: awsconfig.LoadDefaultConfig fails while resolving the config chain during getDetailsForBucket: malformed ~/.aws/config or ~/.aws/credentials (bad INI syntax), a referenced source_profile or role_arn that cannot be assumed, invalid IMDS client configuration options, or an invalid AWS_SDK_LOAD_CONFIG environment setup. Raised only when kops is running on EC2 (isRunningOnEC2 detected EC2 product_uuid) and the bucket region was not already resolvable.
Common situations: Hand-edited ~/.aws/config with syntax errors; profile with role_arn whose trust policy doesn't allow the caller; broken credential_process output; AWS_CONFIG_FILE pointing at a nonexistent/corrupt file; SDK v2 config options conflicting (e.g. bad retry/endpoint settings injected via env).
Related errors
- error initializing AWS client: %v
- building presigned request: %w
- DeviceName not set for volume
- error loading default AWS config: %v
- error populating configuration: %v
AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05).
Data as JSON: /api/errors/c164446ebc38f0dd.
Report an issue: GitHub.