kubernetes/kops · error

error reading target-lifecycle-state from instance metadata:

Error message

error reading target-lifecycle-state from instance metadata: %v

What it means

getAWSConfigurationMode reads the EC2 instance-metadata key target-lifecycle-state to detect warm-pool nodes. If the metadata read fails with any error other than HTTP 404, nodeup returns this wrapped error and aborts. It is thrown because instance metadata was unreachable or returned an unexpected status.

Source

Thrown at upup/pkg/fi/nodeup/command.go:881

	// Check if WarmPool is enabled first, to avoid additional API calls
	if len(c.NodeupConfig.WarmPoolImages) == 0 {
		return "", nil
	}

	// Only worker nodes and apiservers can actually autoscale.
	// We are not adding describe permissions to the other roles
	role := c.BootConfig.InstanceGroupRole
	if !role.HasNode() && !role.HasAPIServer() {
		return "", nil
	}

	targetLifecycleState, err := vfs.Context.ReadFile("metadata://aws/meta-data/autoscaling/target-lifecycle-state")
	if err != nil {
		var awsErr *awshttp.ResponseError
		if errors.As(err, &awsErr) && awsErr.HTTPStatusCode() == http.StatusNotFound {
			return "", nil
		}
		return "", fmt.Errorf("error reading target-lifecycle-state from instance metadata: %v", err)
	}

	if strings.HasPrefix(string(targetLifecycleState), "Warmed:") {
		klog.Info("instance is entering warm pool")
		return model.ConfigurationModeWarming, nil
	} else {
		klog.Info("instance is entering the ASG")
		return "", nil
	}
}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Verify IMDS is enabled and reachable: `curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/autoscaling/target-lifecycle-state`
2. Check instance metadata options on the ASG (http-tokens, http-hop-limit=2, http-endpoint=enabled)
3. Confirm the instance is launched from an ASG with warm pool configured; 404 is tolerated, other statuses are fatal
4. Retry nodeup — transient metadata unavailability during instance launch resolves shortly

token, _ := imdsToken(ctx)
resp, err := client.Get("http://169.254.169.254/latest/meta-data/autoscaling/target-lifecycle-state")
// treat 404 as "no warm pool", any other failure as environment problem to fix first

if err := runNodeup(ctx); err != nil {
  if strings.Contains(err.Error(), "target-lifecycle-state") {
    // wait briefly and retry: IMDS is often transiently unavailable at launch
    time.Sleep(5 * time.Second)
    return runNodeup(ctx)
  }
  return err
}

When it happens

Trigger: vfs read of metadata://aws/meta-data/autoscaling/target-lifecycle-state fails with a non-404 error — metadata service unreachable, HTTP 403 (IMDSv2 token issue), timeout, or 500.

Common situations: Instance metadata options set to require tokens but hop limit blocks nodeup; metadata service disabled on the ASG; networking issue in a launching instance; non-AWS environment where this metadata path does not exist.

Related errors

• failed to load AWS config: %w
• failed to get local-ipv4 address from ec2 metadata: %w
• error loading AWS config: %v
• error reading instance-id from AWS metadata: %v
• failed to get machine type: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/7c69b51a091c4b7d. Report an issue: GitHub.