hashicorp/nomad · error

failed to setup IMDS client: %v

Error message

failed to setup IMDS client: %v

What it means

The AWS fingerprinter creates an EC2 IMDS (Instance Metadata Service) client before probing. f.imdsClient(ctx) builds the AWS SDK IMDS client (resolving region/endpoint config); if construction or configuration fails, the fingerprint returns this wrapped error and AWS attributes are not detected.

Source

Thrown at client/fingerprint/env_aws.go:94

	)
}

func (f *EnvAWSFingerprint) Fingerprint(request *FingerprintRequest, response *FingerprintResponse) error {
	cfg := request.Config

	timeout := AwsMetadataTimeout

	// Check if we should tighten the timeout
	if cfg.ReadBoolDefault(TightenNetworkTimeoutsConfig, false) {
		timeout = 1 * time.Millisecond
	}

	ctx, cancel := context.WithTimeout(context.TODO(), timeout)
	defer cancel()

	imdsClient, err := f.imdsClient(ctx)
	if err != nil {
		return fmt.Errorf("failed to setup IMDS client: %v", err)
	}

	if err := awsProbe(ctx, imdsClient); err != nil {
		return wrapProbeError(err)
	}

	// Keys and whether they should be namespaced as unique. Any key whose value
	// uniquely identifies a node, such as ip, should be marked as unique. When
	// marked as unique, the key isn't included in the computed node class.
	keys := map[string]bool{
		"ami-id":                      false,
		"hostname":                    true,
		"instance-id":                 true,
		"instance-life-cycle":         false,
		"instance-type":               false,
		"local-hostname":              true,
		"local-ipv4":                  true,
		"public-hostname":             true,

View on GitHub (pinned to 482b49bf1a)

Solutions

  1. Verify the instance can reach 169.254.169.254 (curl from the host)
  2. Check AWS_EC2_METADATA_SERVICE_ENDPOINT for typos
  3. Increase the fingerprint timeout if IMDSv2 token acquisition is slow
  4. Clear stray AWS_* env vars leaking into the Nomad agent process
  5. Confirm IMDS hop limit >=1 (or 2 for containers) when IMDSv2 is enforced

Example fix

// before
export AWS_EC2_METADATA_SERVICE_ENDPOINT="http://169.254.169.254:80" // wrong port blocks client setup
// after
unset AWS_EC2_METADATA_SERVICE_ENDPOINT  # let SDK use default
Defensive patterns

Strategy: try-catch

Validate before calling

resp, err := http.Get("http://169.254.169.254/latest/meta-data/", )
// pre-check reachability before fingerprinting
if err != nil { log.Println("IMDS unreachable; skipping aws fingerprint") }

Type guard

func imdsReachable(ctx context.Context) bool { req, _ := http.NewRequestWithContext(ctx, http.MethodPut, "http://169.254.169.254/latest/api/token", nil); _, err := http.DefaultClient.Do(req); return err == nil }

Try / catch

imdsClient, err := f.imdsClient(ctx)
if err != nil {
    var rerr *retry.Error
    if errors.As(err, &rerr) { /* handle retry/timeout */ }
    return fmt.Errorf("failed to setup IMDS client: %w", err)
}

Prevention

When it happens

Trigger: Failure constructing the imds.Client — invalid AWS_* environment configuration interfering, endpoint resolution error, or context deadline exceeded while setting up the client.

Common situations: Non-EC2 environment with AWS env vars set that break IMDS defaults; IMDS endpoint unreachable within timeout (firewall, IMDSv2-only hop limit); misconfigured AWS_EC2_METADATA_SERVICE_ENDPOINT.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/f7e1b5d5d411a47f. Report an issue: GitHub.