{"record":{"id":"a608c9dac574f4b3","repo":"kubernetes/kops","slug":"cannot-mix-egress-values-in-private-or-ipv6-capabl","errorCode":null,"errorMessage":"cannot mix egress values in private or IPv6-capable subnets","messagePattern":"cannot mix egress values in private or IPv6-capable subnets","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/model/awsmodel/network.go","lineNumber":432,"sourceCode":"\n\t\tegress := info.NATSubnets[0].Egress\n\t\tpublicIP := info.NATSubnets[0].PublicIP\n\n\t\tallUnmanaged := true\n\t\tfor _, subnetSpec := range info.NATSubnets {\n\t\t\tif !isUnmanaged(subnetSpec) {\n\t\t\t\tallUnmanaged = false\n\t\t\t}\n\t\t}\n\t\tif allUnmanaged {\n\t\t\tklog.V(4).Infof(\"skipping network configuration in zone %s - all subnets unmanaged\", zone)\n\t\t\tcontinue\n\t\t}\n\n\t\t// Verify we don't have mixed values for egress/publicIP - the code doesn't handle it\n\t\tfor _, subnet := range info.NATSubnets {\n\t\t\tif subnet.Egress != egress {\n\t\t\t\treturn fmt.Errorf(\"cannot mix egress values in private or IPv6-capable subnets\")\n\t\t\t}\n\t\t\tif subnet.PublicIP != publicIP {\n\t\t\t\treturn fmt.Errorf(\"cannot mix publicIP values in private or IPv6-capable subnets\")\n\t\t\t}\n\t\t}\n\n\t\tvar ngw *awstasks.NatGateway\n\t\tvar tgwID *string\n\t\tvar in *awstasks.Instance\n\t\tif egress != \"\" {\n\t\t\tif strings.HasPrefix(egress, \"nat-\") {\n\n\t\t\t\tngw = &awstasks.NatGateway{\n\t\t\t\t\tName:                 new(zone + \".\" + b.ClusterName()),\n\t\t\t\t\tLifecycle:            b.Lifecycle,\n\t\t\t\t\tSubnet:               egressSubnet,\n\t\t\t\t\tID:                   new(egress),\n\t\t\t\t\tAssociatedRouteTable: egressRouteTable,","sourceCodeStart":414,"sourceCodeEnd":450,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/pkg/model/awsmodel/network.go#L414-L450","documentation":"During AWS network model building, kops groups NAT-capable subnets per zone and takes the Egress value from the first subnet as the zone-wide setting. If any other subnet in that zone declares a different egress value, the builder aborts because it cannot provision one egress path (NAT gateway, NAT instance, Transit Gateway, etc.) per zone with conflicting per-subnet settings. This is a cluster-spec validation error raised at `kops update cluster` model-render time, not an AWS API error.","triggerScenarios":"Running `kops update cluster` when two or more subnets in the same zone that route through NAT (private or IPv6-capable subnets) specify different `egress` values in cluster.spec.networkID/subnets config — e.g. one subnet with egress: nat-xxxx and another with egress: External, or one with an eipalloc- ID and another empty.","commonSituations":"Hand-editing the cluster spec to give one private subnet a Transit Gateway egress while siblings still use the default NAT gateway; migrating a cluster gradually from NAT gateways to NAT instances and changing only some subnets; copy-pasting subnet blocks between zones/zones with differing egress fields.","solutions":["Open the cluster spec and make the `egress` field identical for every private/IPv6-capable subnet within the same zone (leave egress empty on all of them for kops-managed NAT gateways).","If per-zone egress is genuinely needed, move the differing subnets into a different zone, since the check is per-zone (infoByZone).","Run `kops edit cluster` (or `kops replace -f`) with the corrected spec, then `kops update cluster` again.","If you intentionally want mixed egress, split the cluster or file a feature request — the builder explicitly does not support mixed values (see comment at network.go:429)."],"exampleFix":"# before (cluster.yaml, same zone)\nsubnets:\n- name: us-east-1a-private\n  zone: us-east-1a\n  type: Private\n  egress: nat-0abc123\n- name: us-east-1a-private2\n  zone: us-east-1a\n  type: Private\n  egress: External\n# after\nsubnets:\n- name: us-east-1a-private\n  zone: us-east-1a\n  type: Private\n  egress: nat-0abc123\n- name: us-east-1a-private2\n  zone: us-east-1a\n  type: Private\n  egress: nat-0abc123","handlingStrategy":"validation","validationCode":"#!/bin/bash\n# Run before `kops update cluster`: ensure all private subnets per zone share one egress value\nkops get cluster -oyaml | awk '\n  /zone:/ {zone=$2} /egress:/ {print zone, $2}' | sort -u |\n  awk '{if (seen[$1] && seen[$1]!=$2) {print \"CONFLICT in zone\", $1; exit 1} seen[$1]=$2}'","typeGuard":"func consistentEgress(subnets []kops.ClusterSubnetSpec, zone string) bool {\n    seen := map[string]bool{}\n    for _, s := range subnets {\n        if s.Zone != zone { continue }\n        seen[s.Egress] = true\n        if len(seen) > 1 { return false }\n    }\n    return true\n}","tryCatchPattern":"err := updateCluster(ctx)\nif err != nil && strings.Contains(err.Error(), \"cannot mix egress values\") {\n    // normalize egress across subnets in the zone, then retry once\n}","preventionTips":["Keep egress empty on all private subnets unless you deliberately reuse an existing NAT/TGW resource.","Template subnet specs from a single variable so egress can never drift between blocks.","Run `kops toolbox template`/dry-run and grep the rendered spec for per-zone egress consistency before applying.","Review `kops get cluster -oyaml` after every manual edit to the subnets section."],"tags":["aws","networking","cluster-spec","validation"],"backgroundTag":"mixed-egress-subnet-config","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}