kubernetes/kops · error

error terminating instances: %v

Error message

error terminating instances: %v

What it means

DeleteInstances calls EC2 TerminateInstances in batches; if the call fails with any code other than InvalidInstanceID.NotFound (which is treated as already-terminated), the error is wrapped as 'error terminating instances: %v'. This happens while tearing down cluster resources during `kops delete cluster` or instance group deletes.

Source

Thrown at pkg/resources/aws/aws.go:355

		}

		klog.Infof("Terminating %d EC2 instances", len(ids))
		request := &ec2.TerminateInstancesInput{
			InstanceIds:    ids,
			SkipOsShutdown: aws.Bool(true),
		}
		if hasXenHyervisor {
			// SkipOsShutdown is not supported on Xen instance types.
			request.SkipOsShutdown = aws.Bool(false)
		}
		ids = []string{}
		hasXenHyervisor = false
		_, err := c.EC2().TerminateInstances(ctx, request)
		if err != nil {
			if awsup.AWSErrorCode(err) == "InvalidInstanceID.NotFound" {
				klog.V(2).Infof("Got InvalidInstanceID.NotFound error terminating instances; will treat as already terminated")
			} else {
				return fmt.Errorf("error terminating instances: %v", err)
			}
		}
	}
	return nil
}

func ListInstances(cloud fi.Cloud, vpcID, clusterName string) ([]*resources.Resource, error) {
	ctx := context.TODO()
	c := cloud.(awsup.AWSCloud)

	klog.V(2).Infof("Querying EC2 instances")
	filters := BuildEC2Filters(cloud)
	filters = append(filters, awsup.NewEC2Filter("vpc-id", vpcID))
	filters = append(filters, awsup.NewEC2Filter("instance-state-name", string(ec2types.InstanceStateNameRunning)))
	request := &ec2.DescribeInstancesInput{
		Filters: filters,
	}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Check the wrapped underlying cause (%v) for the AWS error code and message
2. Verify IAM permissions include ec2:TerminateInstances for the region/account
3. Retry on throttling errors; kops delete can simply be re-run (idempotent)
4. Check AWS health dashboard and region connectivity for outages

err := deleteInstances(ctx, c, ids)
if err != nil {
	if awsup.AWSErrorCode(err) != "" && isThrottlingCode(awsup.AWSErrorCode(err)) {
		time.Sleep(backoff)
		err = deleteInstances(ctx, c, ids) // retry idempotent delete
	}
	if err != nil {
		return fmt.Errorf("terminate failed: %w", err)
	}
}

When it happens

Trigger: EC2 API returns UnauthorizedOperation, InvalidInstanceID.Malformed, throttling (RequestLimitExceeded), or transient 5xx; also network failures between kops and EC2.

Common situations: IAM credentials lacking ec2:TerminateInstances on the instance/region, instances in a state that forbids termination (e.g. protected via termination protection or already gone with a different error), or API rate limits during large deletes.

Related errors

• error deleting ec2 LaunchTemplate %q: %v
• describing instance for arn %q
• error describing instances: %v
• error describing addresses: %v
• error describing volumes: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/ba7f69d191562cd1. Report an issue: GitHub.