kubernetes/kops · error

Could not find '%s' tag from route table

Error message

Could not find '%s' tag from route table

What it means

When the private route table has more than one NAT-gateway route, kOps disambiguates by filtering gateways tagged for the cluster using the route table's TagClusterName tag. If that cluster-name tag is missing from the route table, disambiguation is impossible and this error is thrown.

Source

Thrown at upup/pkg/fi/cloudup/awstasks/natgateway.go:229

			return nil, fmt.Errorf("error finding associated RouteTable to NatGateway: %v", err)
		}

		if rt != nil {
			var natGatewayIDs []*string
			natGatewayIDsSeen := map[string]bool{}
			for _, route := range rt.Routes {
				if route.NatGatewayId != nil && route.State != ec2types.RouteStateBlackhole && !natGatewayIDsSeen[*route.NatGatewayId] {
					natGatewayIDs = append(natGatewayIDs, route.NatGatewayId)
					natGatewayIDsSeen[*route.NatGatewayId] = true
				}
			}

			if len(natGatewayIDs) == 0 {
				klog.V(2).Infof("no NatGateway found in route table %s", *rt.RouteTableId)
			} else if len(natGatewayIDs) > 1 {
				clusterName, ok := routeTable.Tags[awsup.TagClusterName]
				if !ok {
					return nil, fmt.Errorf("Could not find '%s' tag from route table", awsup.TagClusterName)
				}
				filteredNatGateways := []*ec2types.NatGateway{}
				for _, natGatewayID := range natGatewayIDs {
					gw, err := findNatGatewayById(ctx, cloud, fi.ValueOf(natGatewayID))
					if err != nil {
						return nil, err
					}

					if raws.HasOwnedTag(string(ec2types.ResourceTypeNatgateway)+":"+fi.ValueOf(natGatewayID), gw.Tags, clusterName) {
						filteredNatGateways = append(filteredNatGateways, gw)
					}
				}
				if len(filteredNatGateways) == 0 {
					klog.V(2).Infof("no kOps NatGateway found in route table %s", *rt.RouteTableId)
				} else if len(filteredNatGateways) > 1 {
					return nil, fmt.Errorf("found multiple kOps NatGateways in route table %s", *rt.RouteTableId)
				} else {
					return filteredNatGateways[0], nil

View on GitHub (pinned to 4c8573c808)

Solutions

1. Re-add the cluster-name tag to the route table: `aws ec2 create-tags --resources rtb-xxx --tags Key=KubernetesCluster,Value=<cluster-name>`
2. Delete the duplicate nat- routes so only one 0.0.0.0/0 → nat-gateway route remains, avoiding the multi-gateway path entirely
3. Audit tagging policies (including AWS Tag Policies / automation) so kOps tags aren't stripped from EC2 resources

Example fix

// before: cluster tag stripped from route table
// after: restore it
aws ec2 create-tags --resources rtb-0abc --tags Key=KubernetesCluster,Value=mycluster.example.com

for _, rt := range routeTables {
  if _, ok := rt.Tags["KubernetesCluster"]; !ok {
    fmt.Printf("route table %s missing KubernetesCluster tag — restore it before kops update\n", *rt.RouteTableId)
  }
}

func hasClusterTag(rt *RouteTable) bool {
  _, ok := rt.Tags[awsup.TagClusterName]
  return ok
}

ngw, err := findNatGatewayFromRouteTable(ctx, cloud, rt)
if err != nil && strings.Contains(err.Error(), "Could not find") {
  // restore the cluster-name tag on the route table, then retry
}

When it happens

Trigger: len(natGatewayIDs) > 1 (multiple nat- routes on the table) AND routeTable.Tags has no awsup.TagClusterName ('kops.k8s.io/cluster' / 'KubernetesCluster') entry — i.e. multiple NAT gateways plus a stripped/renamed cluster tag.

Common situations: Manual tag edits or third-party tag-enforcement tools removed KubernetesCluster from route tables; multi-cluster VPC reuse where tables were re-tagged; partial cleanup of a deleted cluster's resources reused by another.

Related errors

• found multiple tags for: %v
• found multiple kOps NatGateways in route table %s
• unable to tag NatGateway
• found multiple RouteTables matching tags
• found multiple Volumes with name: %s

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/a94a367fc1bb0081. Report an issue: GitHub.