kubernetes/kops · error

error deleting GCE DNS resource record set %v

Error message

error deleting GCE DNS resource record set %v

What it means

Returned by deleteDNSRecords when the Cloud DNS Changes().Create call carrying the record Deletions fails. kops issues one change batch deleting all cluster A records from the zone; if the API rejects the change, this error is surfaced during cluster deletion. The wrapped error contains the API-level reason (conflict, not-found, permissions).

Source

Thrown at pkg/resources/gce/gce.go:1387

	return resourceTrackers, nil
}

func deleteDNSRecords(cloud fi.Cloud, r []*resources.Resource) error {
	c := cloud.(gce.GCECloud)
	var records []*clouddns.ResourceRecordSet
	var zoneName string

	for _, record := range r {
		r := record.Obj.(*clouddns.ResourceRecordSet)
		zoneName = record.GroupKey
		records = append(records, r)
	}

	change := clouddns.Change{Deletions: records, Kind: "dns#change", IsServing: true}
	_, err := c.CloudDNS().Changes().Create(c.Project(), zoneName, &change)
	if err != nil {
		return fmt.Errorf("error deleting GCE DNS resource record set %v", err)
	}
	return nil
}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Verify the zone still exists and the records are present: gcloud dns record-sets list --zone <zone> --project <project>.
2. Ensure the service account has roles/dns.admin (write) on the project — read-only roles cause 'forbidden' here.
3. If records were already removed by a prior run, re-run discovery/delete; the leftover records simply no longer exist.
4. For precondition/conflict errors, retry after the in-flight change completes; avoid concurrent kops delete runs against the same cluster.

Example fix

// before: parallel deletes conflict
kubectl-ops delete cluster A & kops delete cluster A & wait
// after: single serialized delete
kops delete cluster A --yes

zone, err := dnsService.ManagedZones.Get(project, zoneName).Do()
if err != nil {
    return fmt.Errorf("zone %s not accessible, skip delete change: %w", zoneName, err)
}
rs, err := dnsService.ResourceRecordSets.List(project, zoneName).Do()
if err != nil {
    return err
}
if len(rs.Rrsets) == 0 {
    return nil // nothing to delete; skip Changes().Create
}

var apiErr *googleapi.Error
if errors.As(err, &apiErr) && apiErr.Code == 404 {
    // zone or records already deleted; treat as success/no-op
}

_, err := c.CloudDNS().Changes().Create(project, zoneName, &change)
var apiErr *googleapi.Error
if errors.As(err, &apiErr) {
    if apiErr.Code == 404 || apiErr.Code == 409 {
        // already deleted or concurrent change: re-list and reconcile, don't hard-fail
    } else if apiErr.Code == 403 {
        // missing roles/dns.admin: fix IAM
    }
}

When it happens

Trigger: Changes().Create(project, zoneName, &change{Deletions: records}) fails: the zone no longer exists, records were already deleted/modified (change conflict / precondition failure), insufficient DNS write permission, or API outage.

Common situations: Running kops delete cluster twice in parallel (second run conflicts with already-deleted records); zone deleted out-of-band in the GCP console; service account lacks roles/dns.admin so delete is forbidden; stale discovery data referencing outdated records.

Related errors

• error getting GCE DNS zones %v
• error getting GCE DNS zone data %v
• creating gce IPAM controller: %w
• error building compute API client: %v
• error deleting route53 record %q: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/daedca9ac42047b7. Report an issue: GitHub.