{"record":{"id":"0c5974c3e269b432","repo":"kubernetes/kops","slug":"error-reading-from-aws-metadata-service-v","errorCode":null,"errorMessage":"error reading from AWS metadata service: %v","messagePattern":"error reading from AWS metadata service: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"util/pkg/vfs/context.go","lineNumber":239,"sourceCode":"\n\treturn nil, fmt.Errorf(\"unknown / unhandled path type: %q\", p)\n}\n\n// readAWSMetadata reads the specified path from the AWS EC2 metadata service\nfunc (c *VFSContext) readAWSMetadata(ctx context.Context, path string) ([]byte, error) {\n\tconfig, err := awsconfig.LoadDefaultConfig(ctx)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to load AWS config: %w\", err)\n\t}\n\n\tclient := imds.NewFromConfig(config)\n\n\tif strings.HasPrefix(path, \"/meta-data/\") {\n\t\ts, err := client.GetMetadata(ctx, &imds.GetMetadataInput{\n\t\t\tPath: strings.TrimPrefix(path, \"/meta-data/\"),\n\t\t})\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error reading from AWS metadata service: %v\", err)\n\t\t}\n\t\tdefer s.Content.Close()\n\t\treturn io.ReadAll(s.Content)\n\t}\n\t// There are others (e.g. user-data), but as we don't use them yet let's not expose them\n\treturn nil, fmt.Errorf(\"unhandled aws metadata path %q\", path)\n}\n\n// readHTTPLocation reads an http (or https) url.\n// It returns the contents, or an error on any non-200 response.  On a 404, it will return os.ErrNotExist\n// It will retry a few times on a 500 class error\nfunc (c *VFSContext) readHTTPLocation(httpURL string, httpHeaders map[string]string, opts vfsOptions) ([]byte, error) {\n\tvar body []byte\n\n\tdone, err := RetryWithBackoff(opts.backoff, func() (bool, error) {\n\t\tklog.V(4).Infof(\"Performing HTTP request: GET %s\", httpURL)\n\t\treq, err := http.NewRequest(\"GET\", httpURL, nil)\n\t\tif err != nil {","sourceCodeStart":221,"sourceCodeEnd":257,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/util/pkg/vfs/context.go#L221-L257","documentation":"After loading config, readAWSMetadata calls the EC2 IMDS client GetMetadata for /meta-data/ paths. Any error from the instance metadata service (unreachable endpoint, timeout, throttling, token failure) is wrapped as 'error reading from AWS metadata service'.","triggerScenarios":"ReadFile('metadata://aws/meta-data/<key>') when the IMDS endpoint at 169.254.169.254 is unreachable, IMDSv2 token requests fail, the instance is not on AWS, or the metadata option (e.g. instance-id) is disabled via IMDS settings.","commonSituations":"Running the code locally/off-EC2 where 169.254.169.254 doesn't route; IMDS hop limit or firewall rules blocking link-local traffic; instance metadata access set to 'disabled' on the instance; transient IMDS throttling under heavy polling.","solutions":["Verify IMDS is reachable: `curl -s http://169.254.169.254/latest/meta-data/` from the instance","Check the instance metadata options (aws ec2 describe-instances / modify-instance-metadata-options) and re-enable metadata access if disabled","Confirm you are actually running on an EC2 instance - this scheme only works on AWS","Retry on 5xx/throttling; the http path already retries, but IMDS errors here are returned immediately"],"exampleFix":"// before\n// running kops on a laptop\nkops.ReadFile(\"metadata://aws/meta-data/instance-id\")\n// after\nif onEC2, _ := isEC2(); onEC2 {\n\tkops.ReadFile(\"metadata://aws/meta-data/instance-id\")\n} else {\n\tkops.ReadFile(\"file:///etc/instance-id\")\n}","handlingStrategy":"retry","validationCode":"func imdsReachable() bool {\n\tclient := &http.Client{Timeout: 2 * time.Second}\n\tresp, err := client.Get(\"http://169.254.169.254/latest/meta-data/\")\n\treturn err == nil && resp.StatusCode == 200\n}","typeGuard":"func runningOnEC2() bool { return imdsReachable() }","tryCatchPattern":"var imdsBackoff = wait.Backoff{Duration: 500 * time.Millisecond, Factor: 2, Steps: 6}\ndata, err := vfs.RetryWithBackoff(imdsBackoff, func() (bool, error) {\n\tb, err := vfs.Context.ReadFile(\"metadata://aws/meta-data/instance-id\")\n\tif err != nil {\n\t\tif strings.Contains(err.Error(), \"error reading from AWS metadata service\") {\n\t\t\treturn false, err // retry transient IMDS failures\n\t\t}\n\t\treturn true, err\n\t}\n\treturn true, nil\n})\n_ = data","preventionTips":["Only use metadata://aws URLs on EC2 instances; gate with an on-EC2 probe","Ensure instance metadata options are enabled and hop limit allows containers","Avoid hammering IMDS; cache metadata values once read","Check security groups/host firewall rules don't block 169.254.169.254"],"tags":["aws","imds","metadata","network"],"backgroundTag":"metadata-service-unreachable","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}