What causes "error building node labels: %w" in kubernetes/kops?

Inspect the wrapped inner error (%w) in the full log message to see the underlying cause

error building node labels: %w

Error message

error building node labels: %w

What it means

This wrapper error is raised while building node label configuration (pkg/model/context.go:179) after nodelabels.BuildNodeLabels fails. BuildNodeLabels computes the well-known labels (kops.k8s.io/instancegroup, cluster-autoscaler kops.k8s.io/node-labels, etc.) from the cluster and instance group specs; an invalid spec value (e.g. malformed autoscaler node labels) bubbles up here wrapped with 'error building node labels'.

Source

Thrown at pkg/model/context.go:179

	for k, v := range ig.Spec.CloudLabels {
		labels[k] = v
	}

	if b.Cluster.Spec.CloudProvider.AWS != nil {
		// Apply NTH Labels
		nth := b.Cluster.Spec.CloudProvider.AWS.NodeTerminationHandler
		if nth.IsQueueMode() {
			k := fi.ValueOf(nth.ManagedASGTag)
			if _, ok := labels[k]; !ok && k != "" {
				labels[k] = ""
			}
		}
	}

	// Apply labels for cluster autoscaler node labels
	nodeLabels, err := nodelabels.BuildNodeLabels(b.Cluster, ig)
	if err != nil {
		return nil, fmt.Errorf("error building node labels: %w", err)
	}
	for k, v := range nodeLabels {
		switch b.Cluster.GetCloudProvider() {
		case kops.CloudProviderHetzner:
			labels[hetzner.TagKubernetesNodeLabelPrefix+k] = v
		case kops.CloudProviderGCE:
			// TODO: Do nothing for now while we figure out how to address GCE label length limit of 63
		case kops.CloudProviderLinode:
			// Akamai (Linode) Cloud tags have a 50 character limit
			// Only store the critical kops.k8s.io/instancegroup label
			// Role labels will be derived from the instance role tag by the identifier
			if k == linode.TagKubernetesInstanceGroup {
				labels[k] = v
			}
		default:
			labels[nodeidentityaws.ClusterAutoscalerNodeTemplateLabel+k] = v
		}
	}

View on GitHub (pinned to 4c8573c808)

Solutions

Inspect the wrapped inner error (%w) in the full log message to see the underlying cause
Run 'kops edit ig <name>' and fix spec.wellKnownLabels / node-label entries to be valid 'key=value' pairs
Validate keys against Kubernetes label key/value constraints (63 chars, alphanumeric with -_.)
Re-run 'kops update cluster' after correcting the spec

Example fix

// before (instance group)
wellKnownLabels: "kops.k8s.io/instancegroup=nodes, node-labels="
// after
wellKnownLabels: "kops.k8s.io/instancegroup=nodes,kops.k8s.io/scale-to-zero=true"

Defensive patterns

Strategy: try-catch

Validate before calling

// Validate label entries are key=value and key/value charset-valid before building
for _, kv := range strings.Split(ig.Spec.WellKnownLabels, ",") {
	parts := strings.SplitN(strings.TrimSpace(kv), "=", 2)
	if len(parts) != 2 || parts[0] == "" || parts[1] == "" {
		return fmt.Errorf("malformed label %q in IG %q", kv, ig.Name)
	}
}

Try / catch

nodeLabels, err := nodelabels.BuildNodeLabels(cluster, ig)
if err != nil {
	return fmt.Errorf("error building node labels: %w", err) // inspect wrapped cause in logs
}

Prevention

Use 'kops edit ig' instead of hand-writing wellKnownLabels
Keep label keys within K8s constraints (<=63 chars, [-A-Za-z0-9_.])
Check the wrapped inner error (%w chain) to find the real offending field
Pin the kOps version when generating specs programmatically

When it happens

Trigger: Building a node model for an instance group whose spec contains invalid well-known labels — e.g. instanceGroup.spec.wellKnownLabels with malformed cluster-autoscaler node labels, or a spec the label builder cannot reconcile.

Common situations: Adding cluster-autoscaler node labels via 'kops edit ig' with wrong syntax, version upgrades introducing new label requirements, or templated manifests with malformed label maps.

Related errors

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/fb07628b7d772d1a. Report an issue: GitHub.