kubernetes/kops · error

error creating ResourceRecordSets: %v

Error message

error creating ResourceRecordSets: %v

What it means

In DNSName.RenderAWS, kops issues a Route53 ChangeResourceRecordSets UPSERT for the record; any API rejection is wrapped as 'error creating ResourceRecordSets' (dnsname.go:215). The Route53 error (InvalidChangeBatch, NoSuchHostedZone, AccessDenied, concurrent change) is appended via %v.

Source

Thrown at upup/pkg/fi/cloudup/awstasks/dnsname.go:215

	}

	change := route53types.Change{
		Action:            route53types.ChangeActionUpsert,
		ResourceRecordSet: rrs,
	}

	changeBatch := &route53types.ChangeBatch{}
	changeBatch.Changes = []route53types.Change{change}

	request := &route53.ChangeResourceRecordSetsInput{}
	request.HostedZoneId = e.Zone.ZoneID
	request.ChangeBatch = changeBatch

	klog.V(2).Infof("Updating DNS record %q", *e.ResourceName)

	response, err := t.Cloud.Route53().ChangeResourceRecordSets(context.TODO(), request)
	if err != nil {
		return fmt.Errorf("error creating ResourceRecordSets: %v", err)
	}

	klog.V(2).Infof("Change id is %q", aws.ToString(response.ChangeInfo.Id))

	return nil
}

type terraformRoute53Record struct {
	Name    *string  `cty:"name"`
	Type    *string  `cty:"type"`
	TTL     *string  `cty:"ttl"`
	Records []string `cty:"records"`

	Alias  *terraformAlias          `cty:"alias"`
	ZoneID *terraformWriter.Literal `cty:"zone_id"`
}

type terraformAlias struct {

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Read the wrapped Route53 error (InvalidChangeBatch, NoSuchHostedZone, AccessDenied) and fix that specific cause
  2. If NoSuchHostedZone: recreate/verify the hosted zone and update the cluster spec
  3. If AccessDenied: grant route53:ChangeResourceRecordSets to the kops credentials
  4. If InvalidChangeBatch: verify ResourceName is well-formed and ResourceType (A/AAAA/CNAME) is valid for the alias
  5. Avoid concurrent kops update runs against the same zone

Example fix

// before: read-only Route53 policy
{"Action": ["route53:List*", "route53:Get*"], "Effect": "Allow"}
// after
{"Action": ["route53:List*", "route53:Get*", "route53:ChangeResourceRecordSets"], "Effect": "Allow", "Resource": "*"}
Defensive patterns

Strategy: retry

Validate before calling

// Validate zone + record ownership before issuing the change
zone, err := r53.GetHostedZone(ctx, &route53.GetHostedZoneInput{Id: e.Zone.ZoneID})
if err != nil {
	return fmt.Errorf("zone %s unavailable: %w", aws.ToString(e.Zone.ZoneID), err)
}
zoneName := strings.TrimSuffix(aws.ToString(zone.HostedZone.Name), ".")
if !strings.HasSuffix(strings.TrimSuffix(aws.ToString(e.ResourceName), "."), zoneName) {
	return fmt.Errorf("record %s does not belong to zone %s", aws.ToString(e.ResourceName), zoneName)
}

Try / catch

_, err := r53.ChangeResourceRecordSets(ctx, req)
if err != nil {
	var batch *route53types.InvalidChangeBatch
	if errors.As(err, &batch) {
		// fix record name/type in the spec — do not retry
	}
	var th *route53types.ThrottlingException
	if errors.As(err, &th) {
		// exponential backoff and retry
	}
	return err
}

Prevention

When it happens

Trigger: ChangeResourceRecordSets fails: invalid ResourceName/Type combination in the ChangeBatch, HostedZoneId (e.Zone.ZoneID) wrong or deleted, IAM lacks route53:ChangeResourceRecordSets, malformed alias target (bad hosted zone ID for the NLB), or a conflicting concurrent change.

Common situations: Zone deleted between Find and Apply; record name with invalid characters or trailing-dot mismatch; read-only IAM policy on Route53; alias target referencing an NLB in an incompatible zone; concurrent kops runs colliding on the same record.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/39d37f12f0ec8ecf. Report an issue: GitHub.