kubernetes/kops · error

error deleting instance: %s

Error message

error deleting instance: %s

What it means

This error is returned by deleteInstanceWithID when the Openstack Nova servers.Delete API call fails with any error other than 404 Not Found. It is wrapped in vfs.RetryWithBackoff, so the delete is retried with backoff until it succeeds, is not found, or the backoff window expires (in which case the last error is surfaced wrapped in this message).

Source

Thrown at upup/pkg/fi/cloudup/openstack/instance.go:199

}

func (c *openstackCloud) DeleteInstance(i *cloudinstances.CloudInstance) error {
	return deleteInstance(c, i)
}

func deleteInstance(c OpenstackCloud, i *cloudinstances.CloudInstance) error {
	return deleteInstanceWithID(c, i.ID)
}

func (c *openstackCloud) DeleteInstanceWithID(instanceID string) error {
	return deleteInstanceWithID(c, instanceID)
}

func deleteInstanceWithID(c OpenstackCloud, instanceID string) error {
	done, err := vfs.RetryWithBackoff(deleteBackoff, func() (bool, error) {
		err := servers.Delete(context.TODO(), c.ComputeClient(), instanceID).ExtractErr()
		if err != nil && !isNotFound(err) {
			return false, fmt.Errorf("error deleting instance: %s", err)
		}
		if isNotFound(err) {
			return true, nil
		}
		return false, nil
	})
	if err != nil {
		return err
	} else if done {
		return nil
	} else {
		return wait.ErrWaitTimeout
	}
}

// DeregisterInstance drains a cloud instance and loadbalancers.
func (c *openstackCloud) DeregisterInstance(i *cloudinstances.CloudInstance) error {
	return deregisterInstance(c, i.ID)

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Check the wrapped %s message for the underlying Nova error (401/409/500) and fix that root cause first
  2. Verify the instance state with `openstack server show <id>`; wait for it to leave BUILD/ERROR state or force-delete it manually
  3. Re-authenticate / refresh credentials; confirm OS_* environment or cloud config is valid and not expired
  4. If retries exhausted, delete the instance manually via CLI and re-run the kops operation

Example fix

// before: fail hard on transient delete errors
err := servers.Delete(context.TODO(), c.ComputeClient(), instanceID).ExtractErr()
if err != nil {
    return false, fmt.Errorf("error deleting instance: %s", err)
}
// after: tolerate 409 instance-already-being-deleted as success
err := servers.Delete(context.TODO(), c.ComputeClient(), instanceID).ExtractErr()
if err != nil && !isNotFound(err) {
    if _, ok := err.(gophercloud.ErrDefault409); ok {
        return false, nil // retry, instance is being torn down
    }
    return false, fmt.Errorf("error deleting instance: %s", err)
}
Defensive patterns

Strategy: retry

Validate before calling

// check instance exists and is in a deletable state before calling delete
server, err := cloud.GetInstance(instanceID)
if err != nil { return err }
if server.Status == "DELETING" { return nil } // already going away

Try / catch

// retry with backoff around the kops delete path
backoff := wait.Backoff{Duration: 5 * time.Second, Factor: 2, Steps: 5}
err := wait.ExponentialBackoff(backoff, func() (bool, error) {
    err := cloud.DeleteInstance(inst)
    if err != nil && strings.Contains(err.Error(), "error deleting instance") {
        return false, nil // transient; retry
    }
    return err == nil, err
})

Prevention

When it happens

Trigger: servers.Delete returns a non-NotFound error: the compute endpoint is unreachable, authentication token expired, the instance is in a state that forbids deletion (e.g. BUILD, vm_state ERROR), or a 409/412/500 is returned by Nova.

Common situations: Keystone token expiry mid-operation; deleting an instance that is still being provisioned; OpenStack API outage or nova-compute failure; network partition between kops controller and the OpenStack cloud.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/58365244a82da98e. Report an issue: GitHub.