kubernetes/kops · error

error creating cloud instance group member: %v

Error message

error creating cloud instance group member: %v

What it means

In osBuildCloudInstanceGroup, each Nova instance matching the instance group is converted into a cloudinstances.CloudInstanceGroup member via cg.NewCloudInstance. If that in-memory construction fails (e.g. the member would be a duplicate ID or the group state is inconsistent), kOps wraps it as "error creating cloud instance group member: %v" and aborts building the cloud instance group used by rolling-update/drain operations.

Source

Thrown at upup/pkg/fi/cloudup/openstack/server_group.go:120

		return nil, err
	}
	for _, instance := range instances {
		if !InstanceInClusterAndIG(instance, cluster.Name, ig.Name) {
			continue
		}

		igObservedGeneration := instance.Metadata[INSTANCE_GROUP_GENERATION]
		clusterObservedGeneration := instance.Metadata[CLUSTER_GENERATION]
		observedName := fmt.Sprintf("%s-%s", clusterObservedGeneration, igObservedGeneration)
		generationName := fmt.Sprintf("%d-%d", cluster.GetGeneration(), ig.Generation)

		status := cloudinstances.CloudInstanceStatusUpToDate
		if generationName != observedName || instance.Status == errorStatus {
			status = cloudinstances.CloudInstanceStatusNeedsUpdate
		}
		cm, err := cg.NewCloudInstance(instance.ID, status, nodeMap[instance.ID])
		if err != nil {
			return nil, fmt.Errorf("error creating cloud instance group member: %v", err)
		}

		if instance.Flavor["original_name"] != nil {
			cm.MachineType = instance.Flavor["original_name"].(string)
		}

		ip, err := GetServerFixedIP(&instance, instance.Metadata[TagKopsNetwork])
		if err != nil {
			klog.Warningf("Unable to find fixed ip for %s: %v", instance.Name, err)
		}

		cm.PrivateIP = ip
		cm.Roles = []string{instance.Metadata["KopsRole"]}
		cm.State = cloudinstances.State(instance.Status)
	}
	return cg, nil
}

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Inspect the wrapped error from NewCloudInstance to identify the duplicate/inconsistent member ID.
  2. List the Nova servers with the group's name prefix ('openstack server list --name ^<ig-name>') and delete or fix duplicate/stale instances.
  3. Verify each instance's kops metadata (KopsInstanceGroup/cluster tags) so instances map to exactly one instance group.
  4. Refresh cluster state (kops update/reconcile) so nodeMap and generation metadata are consistent, then re-run the operation.

Example fix

// before: same Nova server tagged for two instance groups
// metadata: {"KopsInstanceGroup": "nodes", "KopsRole": "node"} but listed under 'masters' prefix too
// after: fix instance metadata so it belongs to one group
openstack server set --property KopsInstanceGroup=nodes <server-id>
// or delete the orphaned duplicate instance
openstack server delete <duplicate-server-id>
Defensive patterns

Strategy: validation

Validate before calling

// Detect duplicate Nova instance IDs for the group before building members
seen := map[string]bool{}
for _, inst := range instances {
	if seen[inst.ID] {
		return fmt.Errorf("duplicate Nova instance %s reported for instance group %s; fix stale instances before rolling update", inst.ID, ig.Name)
	}
	seen[inst.ID] = true
}

Try / catch

cg, err := osBuildCloudInstanceGroup(cloud, cluster, ig, nodeMap)
if err != nil {
	if strings.Contains(err.Error(), "error creating cloud instance group member") {
		// member construction failed: inspect and repair instance metadata/duplicates, then retry
		klog.Errorf("failed building group %s: %v — run 'openstack server list --name ^%s' to find stale instances", ig.Name, err, ig.Name)
	}
	return err
}

Prevention

When it happens

Trigger: cg.NewCloudInstance(instance.ID, status, nodeMap[instance.ID]) returns an error — typically when two Nova instances share the same ID within the group, or the underlying CloudInstanceGroup is in a state that forbids adding the member (e.g. adding a detached/unknown member where the API requires an existing group member).

Common situations: Duplicate Nova instance IDs returned by a buggy listing or reused server group; stale kOps state where the same instance appears in two instance groups; instances with inconsistent kops-instance-group metadata causing the same member to be added twice during a rolling update.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/86a280972bb69e75. Report an issue: GitHub.