kubernetes/kops · error

error deleting route53 record %q: %v

Error message

error deleting route53 record %q: %v

What it means

deleteRoute53Records wraps a failed Route53 ChangeResourceRecordSets call (used to delete the batch of records for one resource) with a human-readable record name. Unlike the ELB/EC2 deleters, there is no IsDependencyViolation special-casing — every API error becomes this wrapped error.

Source

Thrown at pkg/resources/aws/aws.go:1795

		names = append(names, resourceTracker.Name)
		changes = append(changes, route53types.Change{
			Action:            route53types.ChangeActionDelete,
			ResourceRecordSet: resourceTracker.Obj.(*route53types.ResourceRecordSet),
		})
	}
	human := strings.Join(names, ", ")
	klog.V(2).Infof("Deleting route53 records %q", human)

	changeBatch := &route53types.ChangeBatch{
		Changes: changes,
	}
	request := &route53.ChangeResourceRecordSetsInput{
		HostedZoneId: zone.Id,
		ChangeBatch:  changeBatch,
	}
	_, err := c.Route53().ChangeResourceRecordSets(ctx, request)
	if err != nil {
		return fmt.Errorf("error deleting route53 record %q: %v", human, err)
	}
	return nil
}

func ListRoute53Records(cloud fi.Cloud, vpcID, clusterName string) ([]*resources.Resource, error) {
	ctx := context.TODO()
	var resourceTrackers []*resources.Resource

	c := cloud.(awsup.AWSCloud)

	// Normalize cluster name, with leading "."
	clusterName = "." + strings.TrimSuffix(clusterName, ".")

	// TODO: If we have the zone id in the cluster spec, use it!
	var zones []route53types.HostedZone
	{
		klog.V(2).Infof("Querying for all route53 zones")

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Check the hosted zone still exists (aws route53 list-hosted-zones) and that kOps targets the right zone ID.
  2. Compare the record set in Route53 with what kOps tries to delete — manually edited records cause InvalidChangeBatch.
  3. Grant route53:ChangeResourceRecordSets on the zone in the IAM policy.
  4. Retry after transient throttling/5xx; Route53 API writes are rate-limited.
  5. Delete conflicting/mismatched records manually, then re-run kOps cleanup.
Defensive patterns

Strategy: validation

Validate before calling

// Verify the zone and the exact record still exist before the ChangeBatch
zones, err := c.Route53().ListHostedZones(ctx, &route53.ListHostedZonesInput{})
// match zone.Id, then:
rrs, err := c.Route53().ListResourceRecordSets(ctx, &route53.ListResourceRecordSetsInput{
    HostedZoneId: zone.Id, StartRecordName: aws.String(name),
})
// if no matching record with same type/value, skip the delete

Try / catch

_, err := c.Route53().ChangeResourceRecordSets(ctx, request)
if err != nil {
    if awsup.AWSErrorCode(err) == "InvalidChangeBatch" { /* record mismatch: verify via ListResourceRecordSets */ }
    if awsup.AWSErrorCode(err) == "NoSuchHostedZone" { return nil /* zone gone: done */ }
    return err
}

Prevention

When it happens

Trigger: ChangeResourceRecordSets fails: InvalidChangeBatch (malformed record or a record doesn't match exactly), NoSuchHostedZone, prior conditional update conflict, AccessDenied on route53:ChangeResourceRecordSets, or throttling.

Common situations: Records were modified manually in Route53 so the delete batch no longer matches; hosted zone deleted externally mid-teardown; DNSSEC/SOA/NS records that can't be deleted; cross-account zone access issues.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/c183933b3edfb68c. Report an issue: GitHub.