kubernetes/kops · error

failed to read %q from ec2 meta-data: %v

Error message

failed to read %q from ec2 meta-data: %v

What it means

After a successful IMDS GetMetadata response, the body is read with io.ReadAll; if reading the response Content stream fails the error is wrapped as 'failed to read %q from ec2 meta-data'. This indicates the HTTP response began but the body could not be fully consumed.

Source

Thrown at upup/pkg/fi/nodeup/nodetasks/prefix.go:140

func getInstanceMetadataList(ctx context.Context, category string) ([]string, error) {
	cfg, err := awsconfig.LoadDefaultConfig(ctx)
	if err != nil {
		return nil, fmt.Errorf("failed to load aws config: %v", err)
	}
	metadata := imds.NewFromConfig(cfg)
	resp, err := metadata.GetMetadata(ctx, &imds.GetMetadataInput{Path: category})
	if err != nil {
		var awsErr *smithyhttp.ResponseError
		if errors.As(err, &awsErr) && awsErr.HTTPStatusCode() == http.StatusNotFound {
			return nil, nil
		} else {
			return nil, fmt.Errorf("failed to get %q from ec2 meta-data: %v", category, err)
		}
	}
	defer resp.Content.Close()
	lines, err := io.ReadAll(resp.Content)
	if err != nil {
		return nil, fmt.Errorf("failed to read %q from ec2 meta-data: %v", category, err)
	}

	var values []string
	for _, line := range strings.Split(string(lines), "\n") {
		line = strings.TrimSpace(line)
		if len(line) > 0 {
			values = append(values, line)
		}
	}

	return values, nil
}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Retry nodeup; transient IMDS stream errors usually clear on retry.
2. Verify nothing on the host intercepts 169.254.169.254 (iptables NAT rules, http_proxy with link-local in no_proxy).
3. Check for host-level network instability (conntrack exhaustion, NIC issues) around bootstrap time.

// verify IMDS reachable and stable before the call
net.DialTimeout("tcp", "169.254.169.254:80", 2*time.Second)

if err != nil && isTransient(err) { return retryWithBackoff(fn, 3) }

When it happens

Trigger: The IMDS HTTP response body errors mid-stream: connection reset, timeout while reading, or the response Content being nil/closed prematurely.

Common situations: Flaky link-local networking under load; IMDS connection timeouts on very busy nodes; proxies or agents intercepting 169.254.169.254 and truncating responses.

Related errors

• reading primary MAC address from ec2 metadata: %w
• failed to read instance metadata response: %w
• failed to get %q from ec2 meta-data: %v
• error querying ec2 metadata service (for region): %v
• failed to load AWS config: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/9dccab155378a3f3. Report an issue: GitHub.