kubernetes/kops · warning

getting AWS region from metadata: %w

Error message

getting AWS region from metadata: %w

What it means

After loading the AWS config, getRegionFromMetadata calls the IMDS client GetRegion to learn the region of the current EC2 instance. If that call fails (unreachable metadata service, timeout, missing token, IMDSv2 hop limit), the error is wrapped as this message. kOps uses this as a fallback to infer the region of the S3 state-store bucket.

Source

Thrown at util/pkg/vfs/s3context.go:356

	ctx, span := tracer.Start(ctx, "getRegionFromMetadata")
	defer span.End()

	// Use an even shorter timeout, to minimize impact when not running on EC2
	// Note that we still retry a few times, this works out a little under a 1s delay
	shortTimeout := &http.Client{
		Timeout: 100 * time.Millisecond,
	}

	config, err := awsconfig.LoadDefaultConfig(ctx, awsconfig.WithHTTPClient(shortTimeout))
	if err != nil {
		return "", fmt.Errorf("failed to load AWS config: %w", err)
	}

	client := imds.NewFromConfig(config)

	metadataRegion, err := client.GetRegion(ctx, &imds.GetRegionInput{})
	if err != nil {
		return "", fmt.Errorf("getting AWS region from metadata: %w", err)
	}

	return metadataRegion.Region, nil
}

func VFSPath(url string) (string, error) {
	if !s3UrlRegexp.MatchString(url) {
		return "", fmt.Errorf("%s is not a valid S3 URL", url)
	}
	groupNames := s3UrlRegexp.SubexpNames()
	result := s3UrlRegexp.FindAllStringSubmatch(url, -1)[0]

	captured := map[string]string{}
	for i, value := range result {
		if value != "" {
			captured[groupNames[i]] = value
		}
	}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Increase the metadata response hop limit to 2 (EC2 console/CLI: modify-instance-metadata-options --http-put-response-hop-limit 2) when running in containers
2. Verify metadata access is enabled on the instance: aws ec2 describe-instances --query '...MetadataOptions'
3. Open access to 169.254.169.254 in firewall/security-group/NetworkPolicy and test: curl -m 1 http://169.254.169.254/latest/meta-data/placement/region
4. Avoid metadata inference entirely: set AWS_REGION in the environment so LoadDefaultConfig supplies the region without IMDS
5. On non-EC2 hosts misreporting EC2 product_uuid, correct the SMBIOS/product_uuid or run kops from outside that environment

Example fix

// before: container cannot get IMDSv2 token (hop limit 1)
$ kops export kubecfg ...
error: getting AWS region from metadata: failed to get IMDSv2 token
// after: raise hop limit on the instance
aws ec2 modify-instance-metadata-options --instance-id i-123 \
  --http-put-response-hop-limit 2 --http-endpoint enabled

// Check IMDS reachability quickly before relying on it
timeout 2 curl -s http://169.254.169.254/latest/api/token -X PUT >/dev/null \
  || export AWS_REGION=us-east-1  # skip IMDS, use explicit region

func isIMDSError(err error) bool {
	return err != nil && strings.Contains(err.Error(), "getting AWS region from metadata")
}

region, err := getRegionFromMetadata(ctx)
if err != nil {
	if static := os.Getenv("AWS_REGION"); static != "" {
		region = static // fallback to explicit environment region
	} else {
		return fmt.Errorf("could not infer AWS region: %w", err)
	}
}

When it happens

Trigger: client.GetRegion(ctx, &imds.GetRegionInput{}) returns an error during getDetailsForBucket: the instance is tagged as EC2 (product_uuid starts with 'ec2') but IMDS is unreachable or slow — the HTTP client has only a 100ms timeout — IMDSv2 token request blocked (hop limit exceeded in containers, metadata option disabled on the instance), or IMDS endpoint firewalled by security group/network policy.

Common situations: Running kops inside a Docker/Kubernetes container on an EC2 host where the hop limit of 1 prevents IMDSv2 token acquisition; EC2 instance launched with instance metadata access disabled; host firewall or NetworkPolicy blocking 169.254.169.254; IMDS slow/overloaded so the 100ms timeout trips repeatedly (the code retries a few times, ~1s total).

Related errors

• error querying ec2 metadata service (for region): %v
• failed to load AWS config: %w
• failed to get local-ipv4 address from ec2 metadata: %w
• getting primary MAC address from ec2 metadata: %w
• reading primary MAC address from ec2 metadata: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/26f38482645d4bd2. Report an issue: GitHub.