hashicorp/nomad · error

failed to setup IMDS client

Error message

failed to setup IMDS client: %v

What it means

The AWS fingerprinter creates an EC2 IMDS (Instance Metadata Service) client before probing. f.imdsClient(ctx) builds the AWS SDK IMDS client (resolving region/endpoint config); if construction or configuration fails, the fingerprint returns this wrapped error and AWS attributes are not detected.

Solutions

  1. Verify the instance can reach 169.254.169.254 (curl from the host)
  2. Check AWS_EC2_METADATA_SERVICE_ENDPOINT for typos
  3. Increase the fingerprint timeout if IMDSv2 token acquisition is slow
  4. Clear stray AWS_* env vars leaking into the Nomad agent process
  5. Confirm IMDS hop limit >=1 (or 2 for containers) when IMDSv2 is enforced

Example fix

// before
export AWS_EC2_METADATA_SERVICE_ENDPOINT="http://169.254.169.254:80" // wrong port blocks client setup
// after
unset AWS_EC2_METADATA_SERVICE_ENDPOINT  # let SDK use default
Defensive patterns

Strategy: try-catch

Validate before calling

resp, err := http.Get("http://169.254.169.254/latest/meta-data/", )
// pre-check reachability before fingerprinting
if err != nil { log.Println("IMDS unreachable; skipping aws fingerprint") }

Type guard

func imdsReachable(ctx context.Context) bool { req, _ := http.NewRequestWithContext(ctx, http.MethodPut, "http://169.254.169.254/latest/api/token", nil); _, err := http.DefaultClient.Do(req); return err == nil }

Try / catch

imdsClient, err := f.imdsClient(ctx)
if err != nil {
    var rerr *retry.Error
    if errors.As(err, &rerr) { /* handle retry/timeout */ }
    return fmt.Errorf("failed to setup IMDS client: %w", err)
}

Prevention

When it happens

Trigger: Failure constructing the imds.Client — invalid AWS_* environment configuration interfering, endpoint resolution error, or context deadline exceeded while setting up the client.

Common situations: Non-EC2 environment with AWS env vars set that break IMDS defaults; IMDS endpoint unreachable within timeout (firewall, IMDSv2-only hop limit); misconfigured AWS_EC2_METADATA_SERVICE_ENDPOINT.

Related errors


AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/f7e1b5d5d411a47f. Report an issue: GitHub.

Appendix: source

Thrown at client/fingerprint/env_aws.go:94

	)
}

func (f *EnvAWSFingerprint) Fingerprint(request *FingerprintRequest, response *FingerprintResponse) error {
	cfg := request.Config

	timeout := AwsMetadataTimeout

	// Check if we should tighten the timeout
	if cfg.ReadBoolDefault(TightenNetworkTimeoutsConfig, false) {
		timeout = 1 * time.Millisecond
	}

	ctx, cancel := context.WithTimeout(context.TODO(), timeout)
	defer cancel()

	imdsClient, err := f.imdsClient(ctx)
	if err != nil {
		return fmt.Errorf("failed to setup IMDS client: %v", err)
	}

	if err := awsProbe(ctx, imdsClient); err != nil {
		return wrapProbeError(err)
	}

	// Keys and whether they should be namespaced as unique. Any key whose value
	// uniquely identifies a node, such as ip, should be marked as unique. When
	// marked as unique, the key isn't included in the computed node class.
	keys := map[string]bool{
		"ami-id":                      false,
		"hostname":                    true,
		"instance-id":                 true,
		"instance-life-cycle":         false,
		"instance-type":               false,
		"local-hostname":              true,
		"local-ipv4":                  true,
		"public-hostname":             true,

View on GitHub (pinned to 482b49bf1a)