kubernetes/kops · error

error deleting instance %q, node %q: %v

Error message

error deleting instance %q, node %q: %v

What it means

kOps deletes (terminates) the cloud instance after draining/detaching it. This error wraps a DeleteInstance failure from the cloud provider for an instance that has an associated node name, and aborts the rolling update for that instance.

Source

Thrown at pkg/instancegroups/instancegroups.go:661

	return nil
}

// deleteInstance deletes an Cloud Instance.
func (c *RollingUpdateCluster) deleteInstance(u *cloudinstances.CloudInstance) error {
	id := u.ID
	nodeName := ""
	if u.Node != nil {
		nodeName = u.Node.Name
	}
	if nodeName != "" {
		klog.Infof("Stopping instance %q, node %q, in group %q (this may take a while).", id, nodeName, u.CloudInstanceGroup.HumanName)
	} else {
		klog.Infof("Stopping instance %q, in group %q (this may take a while).", id, u.CloudInstanceGroup.HumanName)
	}

	if err := c.Cloud.DeleteInstance(u); err != nil {
		if nodeName != "" {
			return fmt.Errorf("error deleting instance %q, node %q: %v", id, nodeName, err)
		}
		return fmt.Errorf("error deleting instance %q: %v", id, err)
	}

	return nil
}

// drainNode drains a K8s node.
func (c *RollingUpdateCluster) drainNode(ctx context.Context, u *cloudinstances.CloudInstance) error {
	if c.K8sClient == nil {
		return fmt.Errorf("K8sClient not set")
	}

	if u.Node == nil {
		return fmt.Errorf("node not set")
	}

	if u.Node.Name == "" {

View on GitHub (pinned to 4c8573c808)

Solutions

1. Re-run rolling update; it re-discovers actual group membership
2. Confirm IAM terminate-instance permissions
3. Check whether the instance was already terminated externally (cloud console)
4. Retry after cloud API throttling

// check instance state before terminating
out, _ := ec2Client.DescribeInstances(&ec2.DescribeInstancesInput{InstanceIds: []*string{&u.ID}})
// only terminate if state is running/pending and still a member of the ASG

func terminable(u *cloudinstances.CloudInstance, state string) bool {
    return u != nil && u.ID != "" && (state == "running" || state == "pending")
}

if err := c.Cloud.DeleteInstance(u); err != nil {
    if isNotFound(err) || isAlreadyTerminated(err) { return nil }
    if isThrottling(err) { /* backoff and retry */ }
    return fmt.Errorf("error deleting instance %q, node %q: %v", id, nodeName, err)
}

When it happens

Trigger: c.Cloud.DeleteInstance(u) errors while nodeName != "" — e.g. AWS TerminateInstanceInAutoScalingGroup rejected because the instance was already terminated or not in the ASG, IAM denial, or API throttling.

Common situations: Instance terminated concurrently by cluster-autoscaler or manually; spot instance reclaimed mid-roll; IAM missing terminate permission; API rate limits during large rolls.

Related errors

• error deleting instance %q: %v
• DeleteGroup not implemented on azureCloud
• error detaching instance %q, node %q: %v
• error detaching instance %q: %v
• error deleting Akamai (Linode) instance %s(%s): %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/efc277b2248014ff. Report an issue: GitHub.