kubernetes/kops · error

found server ( ) with unexpected state

Error message

found server %s (%s) with unexpected state: %q

What it means

IdentifyNode resolves a Scaleway instance into kOps node identity labels. After fetching the server, it requires the instance to be in Running or Starting state; otherwise node identity cannot be reliably determined. This error means the node being identified is in a transitional or abnormal lifecycle state.

Solutions

1. Wait for the instance to reach Running state and let kubelet retry node registration
2. Check the server state in the Scaleway console/API (scaleway instance get) and start it if stopped
3. Investigate why the instance is not running: billing, quota, failed boot, or manual shutdown
4. If the instance should never register, remove it from the cluster / delete the machine

Example fix

// before: server is stopped
server.State = "archived"
// after: ensure the server is running before node registration
scw instance server-action action=start server-id=<id>
# or wait: until server.State == "running"

// Go: pre-check server state via Scaleway API
api := instance.NewAPI(client)
srv, err := api.GetServer(&instance.GetServerRequest{ServerID: id, Zone: zone}, scw.WithContext(ctx))
if err == nil && srv != nil && srv.Server.State != instance.ServerStateRunning && srv.Server.State != instance.ServerStateStarting {
    return fmt.Errorf("defer registration: server %s in state %s", id, srv.Server.State)
}

func serverIsRunnable(s *instance.Server) bool {
    return s != nil && (s.State == instance.ServerStateRunning || s.State == instance.ServerStateStarting)
}

When it happens

Trigger: A Scaleway server whose State is not Running/Starting (e.g. stopped, stopping, error, archived) calls IdentifyNode during node registration.

Common situations: Node rebooting or shutting down during cluster bring-up; a stopped/terminated instance whose cloud provider metadata still routes to the identifier; Scaleway API eventual consistency returning a server still provisioning; instances killed by spot/preemptible reclaim.

Related errors

• creating client for Scaleway NodeIdentifier
• found instance , but status is
• providerID not set for node
• providerID not recognized for node
• unexpected format for server id

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/4a171c756b36966e. Report an issue: GitHub.

Appendix: source

Thrown at pkg/nodeidentity/scaleway/identify.go:100

	// If caching is enabled try pulling nodeidentity.Info from cache before doing a Scaleway API call.
	if i.cacheEnabled {
		obj, exists, err := i.cache.GetByKey(serverID)
		if err != nil {
			klog.Warningf("Nodeidentity info cache lookup failure: %v", err)
		}
		if exists {
			return obj.(*nodeidentity.Info), nil
		}
	}

	server, err := i.getServer(ctx, serverID)
	if err != nil {
		return nil, err
	}

	if server.State != instance.ServerStateRunning && server.State != instance.ServerStateStarting {
		return nil, fmt.Errorf("found server %s (%s) with unexpected state: %q", server.Name, server.ID, server.State)
	}

	labels := map[string]string{}
	role := scaleway.InstanceRoleFromTags(server.Tags)
	switch kops.InstanceGroupRole(role) {
	case kops.InstanceGroupRoleControlPlane:
		labels[nodelabels.RoleLabelControlPlane20] = ""
	case kops.InstanceGroupRoleNode:
		labels[nodelabels.RoleLabelNode16] = ""
	case kops.InstanceGroupRoleAPIServer:
		labels[nodelabels.RoleLabelAPIServer16] = ""
	default:
		klog.Warningf("Unknown node role %q for server %s(%s)", role, server.Name, server.ID)
	}

	info := &nodeidentity.Info{
		InstanceID: serverID,
		Labels:     labels,

View on GitHub (pinned to 4c8573c808)