{"record":{"id":"0948ce1bcd99de84","repo":"hashicorp/nomad","slug":"failed-to-query-aws-metadata-w","errorCode":null,"errorMessage":"failed to query AWS metadata: %w","messagePattern":"failed to query AWS metadata: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"client/fingerprint/env_aws.go","lineNumber":284,"sourceCode":"\t\tconfig.WithRetryMaxAttempts(0),\n\t)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\timdsClient := imds.NewFromConfig(cfg, func(o *imds.Options) {\n\t\t// endpoint should only be overridden for testing\n\t\tif f.endpoint != \"\" {\n\t\t\to.Endpoint = f.endpoint\n\t\t}\n\t})\n\treturn imdsClient, nil\n}\n\nfunc awsProbe(ctx context.Context, client *imds.Client) error {\n\tresp, err := client.GetMetadata(ctx, &imds.GetMetadataInput{Path: \"ami-id\"})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to query AWS metadata: %w\", err)\n\t}\n\n\ts, err := readMetadataResponse(resp)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read respose: %w\", err)\n\t}\n\n\tif s == \"\" {\n\t\treturn errors.New(\"empty response from AWS metadata\")\n\t}\n\n\treturn nil\n}\n\n// readImdsResponse reads and formats the IMDS response\n// and most importantly, closes the io.ReadCloser\nfunc readMetadataResponse(resp *imds.GetMetadataOutput) (string, error) {\n\tdefer resp.Content.Close()","sourceCodeStart":266,"sourceCodeEnd":302,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/fingerprint/env_aws.go#L266-L302","documentation":"awsProbe queries IMDS for the ami-id path to confirm the node runs on EC2. If client.GetMetadata returns any error (transport failure, 404/403, IMDSv2 token denial, timeout), the error is wrapped with this message and the AWS fingerprint treats the host as not-AWS or unreachable.","triggerScenarios":"IMDS unreachable (not on EC2, network firewall blocking link-local, iptables rules), IMDSv2 hop-limit too low, request timeout, or metadata service disabled on the instance.","commonSituations":"Running Nomad in a non-AWS environment where the fingerprinter still probes; security-hardened images disabling IMDS; container networking dropping 169.254.169.254; transient network failures during startup.","solutions":["Test curl http://169.254.169.254/latest/meta-data/ami-id from the host","Raise IMDSv2 hop limit (aws ec2 modify-instance-metadata-options --http-put-response-hop-limit 2)","Remove/disable the aws fingerprint in client options if not on EC2","Check firewall/iptables rules for link-local traffic","Retry — transient metadata service outages resolve on agent restart"],"exampleFix":"# before: fingerprint fails, node not detected as AWS\n# after: allow IMDSv2 through container bridge\naws ec2 modify-instance-metadata-options --instance-id i-123 --http-put-response-hop-limit 2 --http-tokens required","handlingStrategy":"retry","validationCode":"ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)\nreq, _ := http.NewRequestWithContext(ctx, \"GET\", \"http://169.254.169.254/latest/meta-data/ami-id\", nil)\nreq.Header.Set(\"X-aws-ec2-metadata-token-ttl-seconds\", \"21600\")\nif _, err := http.DefaultClient.Do(req); err != nil { /* skip aws fingerprint */ }","typeGuard":null,"tryCatchPattern":"err := awsProbe(ctx, imdsClient)\nif err != nil {\n    var rerr *retry.Error\n    if errors.As(err, &rerr) && rerr.HTTPStatusCode() == 404 { /* not EC2 — disable fingerprint */ }\n    return wrapProbeError(err)\n}","preventionTips":["Retry the metadata query with backoff on startup","Disable the aws fingerprint on non-EC2 hosts via allowlist","Monitor IMDSv2 hop-limit settings in your AMI baseline"],"tags":["aws","imds","network"],"backgroundTag":"metadata-service-unreachable","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}