kubernetes/kops · critical

querying IMDS %s: %w

Error message

querying IMDS %s: %w

What it means

queryIMDS wraps errors from imdsHTTPClient.Do(req) with "querying IMDS <path>": the HTTP request to the Azure Instance Metadata Service at http://169.254.169.254 did not complete (transport-level failure). This is the classic "IMDS unreachable" family — DNS/connection/routing/timeouts on the link-local address.

Source

Thrown at upup/pkg/fi/cloudup/azure/azuremetadata/imds.go:80

}

// queryIMDS queries an Azure IMDS endpoint and unmarshals the JSON response.
// https://learn.microsoft.com/en-us/azure/virtual-machines/instance-metadata-service
func queryIMDS(ctx context.Context, path string, params url.Values, result any) error {
	req, err := http.NewRequestWithContext(ctx, "GET", imdsBaseURL+path, nil)
	if err != nil {
		return fmt.Errorf("creating IMDS request: %w", err)
	}
	req.Header.Add("Metadata", "True")

	params.Set("api-version", imdsAPIVersion)
	req.URL.RawQuery = params.Encode()

	klog.V(4).Infof("Azure IMDS query: %q", req.URL.String())

	resp, err := imdsHTTPClient.Do(req)
	if err != nil {
		return fmt.Errorf("querying IMDS %s: %w", path, err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		return fmt.Errorf("querying IMDS %s: status %d", path, resp.StatusCode)
	}

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return fmt.Errorf("reading IMDS response: %w", err)
	}
	klog.V(4).Infof("Azure IMDS response: %d bytes", len(body))

	if err := json.Unmarshal(body, result); err != nil {
		return fmt.Errorf("unmarshalling IMDS response: %w", err)
	}

	return nil

View on GitHub (pinned to 4c8573c808)

Solutions

1. Confirm IMDS reachability with curl -H Metadata:true 'http://169.254.169.254/metadata/instance/compute?api-version=2025-04-07&format=json'
2. If off-Azure, don't run the IMDS-backed authenticator — use the appropriate authenticator for the environment
3. Retry with backoff if transient (boot-time race, throttling)
4. Check iptables/NSG rules and routes for the link-local prefix

// Cheap pre-check that the link-local IMDS address is routable
conn, err := net.DialTimeout("tcp", "169.254.169.254:80", 3*time.Second)
if err != nil { return fmt.Errorf("IMDS not reachable from this host: %w", err) }
conn.Close()

// Distinguish transport failure and retry with capped backoff
var lastErr error
for i := 0; i < 5; i++ {
    token, err := auth.CreateToken(body)
    if err == nil { return token, nil }
    lastErr = err
    if errors.Is(err, context.DeadlineExceeded) || isNetTimeout(err) {
        time.Sleep(time.Duration(1<<i) * time.Second)
        continue
    }
    break
}
return "", lastErr

When it happens

Trigger: Any queryIMDS call (from QueryComputeInstanceMetadata or queryIMDSAttestedDocument) where the client fails: connection refused/timeout (10s client timeout), network unreachable, context cancellation, or TLS/proxy misconfig (proxy is explicitly disabled).

Common situations: Non-Azure environments (local dev, CI, on-prem, other clouds) where 169.254.169.254 is unroutable; NSG/firewall blocking link-local; IMDS temporarily unavailable at VM boot; heavily throttled IMDS dropping connections.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• querying instance metadata: %w
• reading IMDS response: %w
• error querying ec2 metadata service (for region): %v
• failed to load AWS config: %w
• failed to get local-ipv4 address from ec2 metadata: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/fa1681b915693992. Report an issue: GitHub.