kubernetes/kops · error

failed to read instance metadata response: %w

Error message

failed to read instance metadata response: %w

What it means

GetMetadata succeeded but reading the response body stream (result.Content) with io.ReadAll failed, so nodeup cannot convert the body into the instance-type string. This wraps the underlying io read error.

Source

Thrown at upup/pkg/fi/nodeup/command.go:427

func getMachineType(ctx context.Context) (string, error) {
	config, err := awsconfig.LoadDefaultConfig(ctx)
	if err != nil {
		return "", fmt.Errorf("failed to load AWS config: %w", err)
	}

	metadata := imds.NewFromConfig(config)

	// Get the actual instance type by querying the EC2 instance metadata service.
	result, err := metadata.GetMetadata(ctx, &imds.GetMetadataInput{
		Path: "instance-type",
	})
	if err != nil {
		return "", fmt.Errorf("failed to get instance metadata type: %w", err)
	}
	defer result.Content.Close()
	instanceTypeName, err := io.ReadAll(result.Content)
	if err != nil {
		return "", fmt.Errorf("failed to read instance metadata response: %w", err)
	}
	return string(instanceTypeName), err
}

func completeWarmingLifecycleAction(ctx context.Context, cloud *awsup.Cloud, modelContext *model.NodeupModelContext) error {
	asgName := modelContext.BootConfig.InstanceGroupName + "." + modelContext.NodeupConfig.ClusterName
	hookName := "kops-warmpool"
	hooks, err := cloud.DescribeLifecycleHooks(ctx, &autoscaling.DescribeLifecycleHooksInput{
		AutoScalingGroupName: &asgName,
		LifecycleHookNames:   []string{hookName},
	})
	if err != nil {
		return fmt.Errorf("failed to find lifecycle hook %q: %w", hookName, err)
	}

	if len(hooks.LifecycleHooks) > 0 {
		klog.Info("Found ASG lifecycle hook")
		_, err := cloud.CompleteLifecycleAction(ctx, &autoscaling.CompleteLifecycleActionInput{

View on GitHub (pinned to 4c8573c808)

Solutions

1. Retry nodeup; this is typically transient and a fresh run re-reads the metadata cleanly.
2. Remove any transparent proxy/interception rules on traffic to 169.254.169.254.
3. Check IMDS response packet limits (--http-put-response-hop-limit / MTU issues on the metadata link).
4. If persistent, capture traffic to the metadata endpoint to identify the interfering component.

body, err := io.ReadAll(result.Content)
if err != nil {
    if errors.Is(err, io.ErrUnexpectedEOF) || errors.Is(err, syscall.ECONNRESET) {
        // transient stream abort: retry the whole metadata call once
    }
    return fmt.Errorf("failed to read instance metadata response: %w", err)
}

When it happens

Trigger: io.ReadAll(result.Content) errors after a successful metadata.GetMetadata call — the response stream was aborted or reset mid-read (connection reset, proxy interference, timeout).

Common situations: Intermediary (proxy or security agent) truncating link-local HTTP traffic; IMDS connection reset under heavy concurrent metadata queries at boot; buggy transparent MITM firewall on the node.

Related errors

• failed to load AWS config: %w
• failed to get local-ipv4 address from ec2 metadata: %w
• finding primary network interface: %w
• loading AWS config: %w
• reading primary MAC address from ec2 metadata: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/ba23e9805d11abb8. Report an issue: GitHub.