{"record":{"id":"cd9fd0867ce5b24f","repo":"kubernetes/kops","slug":"error-applying-dns-changeset-for-zone-s-v","errorCode":null,"errorMessage":"error applying DNS changeset for zone %s: %v","messagePattern":"error applying DNS changeset for zone (.+?): (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"dns-controller/pkg/dns/dnscontroller.go","lineNumber":327,"sourceCode":"\t\tnewValues := newValueMap[k]\n\t\tif newValues == nil {\n\t\t\terr := op.deleteRecords(k)\n\t\t\tif err != nil {\n\t\t\t\tklog.Infof(\"error deleting records for %s: %v\", k, err)\n\t\t\t\terrors = append(errors, err)\n\t\t\t}\n\t\t}\n\t}\n\n\tfor key, changeset := range op.changesets {\n\t\tif changeset.IsEmpty() {\n\t\t\tcontinue\n\t\t}\n\n\t\tklog.V(2).Infof(\"Applying DNS changeset for zone %s\", key)\n\t\tif err := changeset.Apply(ctx); err != nil {\n\t\t\tklog.Warningf(\"error applying DNS changeset for zone %s: %v\", key, err)\n\t\t\terrors = append(errors, fmt.Errorf(\"error applying DNS changeset for zone %s: %v\", key, err))\n\t\t}\n\t}\n\n\tif len(errors) != 0 {\n\t\treturn errors[0]\n\t}\n\n\t// Success!  Store the snapshot as our new baseline\n\tc.mutex.Lock()\n\tdefer c.mutex.Unlock()\n\tc.lastSuccessfulSnapshot = snapshot\n\treturn nil\n}\n\nfunc (c *DNSController) RemoveRecordsImmediate(records []Record) error {\n\tctx := context.TODO()\n\n\top, err := newDNSOp(c.zoneRules, c.dnsCache)","sourceCodeStart":309,"sourceCodeEnd":345,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/dns-controller/pkg/dns/dnscontroller.go#L309-L345","documentation":"runOnce applies the accumulated changeset per zone via changeset.Apply(ctx). If a provider (Route53, CloudDNS, etc.) rejects the change batch, the error is logged as a warning, aggregated, and the first zone's error is returned wrapped as 'error applying DNS changeset for zone %s: %v'. It signals the provider refused an update batch for that hosted zone.","triggerScenarios":"changeset.Apply(ctx) returns a provider error for a zone: rate limiting/throttling, invalid record payload, permission denied on the hosted zone, or transient API failure.","commonSituations":"AWS Route53 throttling (ThrottlingException) under many pods/services updating at once; IAM policy missing route53:ChangeResourceRecordSets; record TTL/value constraints; DNS API outage.","solutions":["Read the wrapped inner error: if throttled, the runWatcher loop will retry on the next tick; reduce update frequency or batch size.","Fix IAM permissions for the controller's identity on the specific hosted zones (route53:ChangeResourceRecordSets, list/get zones).","Verify the zone still exists and credentials are valid; re-check dns-provider flags and zone filtering (--zone/--zoneid).","For persistent payload errors, inspect the record values the controller derived from the service/ingress annotation."],"exampleFix":"// before: broad IAM policy lacking route53 change rights\n// after: attach to the controller's role\n{\n  \"Effect\": \"Allow\",\n  \"Action\": [\"route53:ChangeResourceRecordSets\", \"route53:ListResourceRecordSets\", \"route53:ListHostedZones\"],\n  \"Resource\": [\"arn:aws:route53:::hostedzone/YOURZONEID\"]\n}","handlingStrategy":"retry","validationCode":"// pre-flight: verify the zone is reachable and credentials work\nzones, err := provider.Zones()\nif err != nil {\n    return fmt.Errorf(\"cannot list zones before applying changesets: %w\", err)\n}\n_ = zones","typeGuard":null,"tryCatchPattern":"if err := changeset.Apply(ctx); err != nil {\n    var throttled bool\n    if strings.Contains(err.Error(), \"Throttling\") {\n        throttled = true\n    }\n    if throttled {\n        time.Sleep(backoff) // then re-apply or wait for next tick\n    } else {\n        klog.Errorf(\"zone %s changeset rejected: %v\", key, err)\n    }\n}","preventionTips":["Grant route53:ChangeResourceRecordSets on exactly the hosted zones the controller manages.","Watch for Throttling exceptions and widen the watcher interval under heavy churn.","Keep record payloads valid (values, TTLs) and verify zone IDs unchanged after zone recreation."],"tags":["dns","route53","changeset","dns-controller","aws"],"backgroundTag":"dns-changeset-apply-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T12:17:11.808Z"}