kubernetes/kops · error

error on getting VM ScaleSet: %s

Error message

error on getting VM ScaleSet: %s

What it means

After a cache miss, IdentifyNode fetches the VM's tags via azureClient.getVMTags to derive cluster/instance-group labels. When that Azure API call fails, the underlying error is wrapped as "error on getting VM ScaleSet: %s". The message is historical (it may fire for plain VMs too); the root cause is in the wrapped Azure SDK error (auth, network, throttling, or not-found).

Source

Thrown at pkg/nodeidentity/azure/identify.go:98

	vmName, err := getVMNameFromProviderID(providerID)
	if err != nil {
		return nil, err
	}

	// If caching is enabled, try pulling nodeidentity.Info from the cache before doing an API call.
	if i.cacheEnabled {
		obj, exists, err := i.cache.GetByKey(vmName)
		if err != nil {
			klog.Warningf("Nodeidentity info cache lookup failure: %v", err)
		}
		if exists {
			return obj.(*nodeidentity.Info), nil
		}
	}

	tags, err := i.azureClient.getVMTags(ctx, providerID)
	if err != nil {
		return nil, fmt.Errorf("error on getting VM ScaleSet: %s", err)
	}

	labels := map[string]string{}
	for k, v := range tags {
		if k == azure.TagClusterName && v != nil {
			labels[kops.LabelClusterName] = *v
		}
		if k == InstanceGroupNameTag && v != nil {
			labels[kops.NodeLabelInstanceGroup] = *v
		}
		if strings.HasPrefix(k, azure.TagNameRolePrefix) {
			role := strings.TrimPrefix(k, azure.TagNameRolePrefix)
			switch role {
			case kops.InstanceGroupRoleControlPlane.ToLowerString():
				labels[nodelabels.RoleLabelControlPlane20] = ""
			case "master":
				labels[nodelabels.RoleLabelControlPlane20] = ""
			case kops.InstanceGroupRoleNode.ToLowerString():

View on GitHub (pinned to 4c8573c808)

Solutions

1. Inspect the wrapped %s detail: fix Azure auth (az credential validity, service principal roles Reader on the VM/VMSS resource group).
2. Confirm network egress from the node/control plane to management.azure.com:443.
3. Check azure configuration in the cluster spec (subscriptionID, tenantID, resourceGroup) matches where the VMs live.
4. For 429 throttling, add backoff/retry or rely on the identity cache; retry IdentifyNode after the transient failure.

if err := azureClient.verifyCredentials(ctx); err != nil {
    return fmt.Errorf("azure credentials invalid before VM tag lookup: %w", err)
}

info, err := identifier.IdentifyNode(ctx, node)
if err != nil && strings.Contains(err.Error(), "error on getting VM ScaleSet") {
    // transient ARM/auth failure: backoff and retry
    return retryWithBackoff(3, func() error {
        _, err = identifier.IdentifyNode(ctx, node)
        return err
    })
}

When it happens

Trigger: IdentifyNode called on an azure:// node with an empty identity cache and getVMTags fails: invalid Azure credentials/service principal, network egress blocked to Azure ARM endpoints, ARM rate limiting (429), the VM was deleted, or wrong subscription configuration.

Common situations: Expired or rotated service-principal credentials; cluster VNet lacking outbound access to management.azure.com; Azure API throttling during scale-up storms of VMSS instances; misconfigured azure.conf (subscription ID / tenant mismatch) in kOps cluster spec.

Related errors

• node identity is required
• unable to check for instanceGroup: %v
• getting rest config: %w
• expected azureblob:// URL, got %q
• creating identity: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/da54c17a3e5fbeec. Report an issue: GitHub.