kubernetes/kops · error

error deleting ngw %q: %v

Error message

error deleting ngw %q: %v

What it means

DeleteNatGateway wraps a failed EC2 DeleteNatGateway call with the resource name. Dependency violations are returned unwrapped so the resource tracker can retry after dependents (like the EIP) are gone; all other AWS errors surface here.

Source

Thrown at pkg/resources/aws/aws.go:1766

	return nil
}

func DeleteNatGateway(cloud fi.Cloud, t *resources.Resource) error {
	ctx := context.TODO()
	c := cloud.(awsup.AWSCloud)

	id := t.ID

	klog.V(2).Infof("Removing NatGateway %s", t.Name)
	request := &ec2.DeleteNatGatewayInput{
		NatGatewayId: &id,
	}
	_, err := c.EC2().DeleteNatGateway(ctx, request)
	if err != nil {
		if IsDependencyViolation(err) {
			return err
		}
		return fmt.Errorf("error deleting ngw %q: %v", t.Name, err)
	}
	return nil
}

func deleteRoute53Records(ctx context.Context, cloud fi.Cloud, zone route53types.HostedZone, resourceTrackers []*resources.Resource) error {
	c := cloud.(awsup.AWSCloud)

	var changes []route53types.Change
	var names []string
	for _, resourceTracker := range resourceTrackers {
		names = append(names, resourceTracker.Name)
		changes = append(changes, route53types.Change{
			Action:            route53types.ChangeActionDelete,
			ResourceRecordSet: resourceTracker.Obj.(*route53types.ResourceRecordSet),
		})
	}
	human := strings.Join(names, ", ")
	klog.V(2).Infof("Deleting route53 records %q", human)

View on GitHub (pinned to 4c8573c808)

Solutions

1. Retry the deletion; NAT gateway deletion is asynchronous so a subsequent pass usually succeeds.
2. Confirm with aws ec2 describe-nat-gateways whether it is already deleted/deleting.
3. Grant ec2:DeleteNatGateway in the caller's IAM policy.
4. Check the wrapped error detail for throttling and add backoff/concurrency limits.
5. Verify the NAT gateway ID/region is correct if AuthFailure or NotFound persists.

ngws, err := c.EC2().DescribeNatGateways(ctx, &ec2.DescribeNatGatewaysInput{NatGatewayIds: []string{*t.ID}})
if err != nil || len(ngws.NatGateways) == 0 || ngws.NatGateways[0].State == "deleted" { /* skip */ }

err := DeleteNatGateway(cloud, r)
if err != nil {
    if awsup.AWSErrorCode(err) == "InvalidNatGatewayID.NotFound" { return nil }
    return err // transient errors retried on next reconciliation pass
}

When it happens

Trigger: DeleteNatGateway fails with AccessDenied (missing ec2:DeleteNatGateway), InvalidNatGatewayID.NotFound (already deleted externally), throttling, or a transient 5xx during cluster teardown.

Common situations: NAT gateway already deleted manually or by another tool while kOps lists-then-deletes; IAM policy gaps; throttling in accounts with many NAT gateways across AZs.

Related errors

• error terminating instances: %v
• error deleting KeyPair %q: %v
• error describing RouteTables: %v
• error describing NatGateways: %v
• error deleting EgressOnlyInternetGateway %q: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/06da9832eb3fa57b. Report an issue: GitHub.