kubernetes/kops · error

error deleting volume: %v

Error message

error deleting volume: %v

What it means

kops wraps any non-404 error from the Cinder block-storage volume delete call (upup/pkg/fi/cloudup/openstack/volume.go:143) inside deleteVolume, which retries with backoff via vfs.RetryWithBackoff. It means the OpenStack Block Storage API rejected or failed the DELETE /volumes/{id} request for a reason other than the volume already being gone. The underlying gophercloud error (auth, quota, state conflict, 5xx) is preserved in %v.

Source

Thrown at upup/pkg/fi/cloudup/openstack/volume.go:143

	})
	if err != nil {
		return err
	} else if done {
		return nil
	} else {
		return wait.ErrWaitTimeout
	}
}

func (c *openstackCloud) DeleteVolume(volumeID string) error {
	return deleteVolume(c, volumeID)
}

func deleteVolume(c OpenstackCloud, volumeID string) error {
	done, err := vfs.RetryWithBackoff(deleteBackoff, func() (bool, error) {
		err := cinder.Delete(context.TODO(), c.BlockStorageClient(), volumeID, cinder.DeleteOpts{}).ExtractErr()
		if err != nil && !isNotFound(err) {
			return false, fmt.Errorf("error deleting volume: %v", err)
		}
		if isNotFound(err) {
			return true, nil
		}
		return false, nil
	})
	if err != nil {
		return err
	} else if done {
		return nil
	} else {
		return wait.ErrWaitTimeout
	}
}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Check the wrapped %v error for the HTTP status; if 409, detach the volume (or wait for the instance to terminate) and rerun `kops delete cluster` / the delete operation.
2. List snapshots/backups referencing the volume with `openstack volume snapshot list` and delete them before deleting the volume.
3. Verify credentials and project scope in the OpenStack cloud config (OS_* env or cloud.conf) and that the token has volume:v2 delete permissions.
4. If the volume is stuck, use `openstack volume set --state available <id>` (admin) or detach via Nova, then delete manually and rerun kops.
5. For persistent 5xx, check the Cinder service health / OpenStack provider status and retry later.

Example fix

// before: deleting while volume still attached
cloud.DeleteVolume(volumeID)
// after: wait for detachment (or detach explicitly) before delete
volume, _ := cloud.GetVolume(volumeID)
if len(volume.Attachments) > 0 {
    return fmt.Errorf("volume %s still attached; retry after detach", volumeID)
}
return cloud.DeleteVolume(volumeID)

// check volume state before delete
v, err := cloud.GetVolume(volumeID)
if err == nil && len(v.Attachments) > 0 {
    return fmt.Errorf("volume %s attached to %v; detach first", volumeID, v.Attachments)
}

func isNotFoundErr(err error) bool {
    var gerr gophercloud.ErrUnexpectedResponseCode
    return errors.As(err, &gerr) && gerr.Actual == http.StatusNotFound
}

err := cloud.DeleteVolume(volumeID)
if err != nil {
    if strings.Contains(err.Error(), "409") || strings.Contains(err.Error(), "Conflict") {
        // volume busy: detach/wait and retry
    } else if isNotFoundErr(err) {
        return nil // already gone
    }
    return err
}

When it happens

Trigger: Cinder returns 409 Conflict because the volume is attached to an instance or has snapshots/dependencies; 400 for a malformed volume ID; 401/403 from expired or insufficient Keystone credentials; transient 500/503 from the cinder service (retried until backoff steps exhausted).

Common situations: Tearing down a cluster while a volume is still attached to a node that has not fully terminated; deleting a cluster whose volumes have existing Cinder snapshots or backups; stale/revoked OpenStack credentials mid-teardown; Cinder service degraded or over capacity.

Related errors

• error deleting Akamai (Linode) volume %s(%s): %w
• wait time exceeded during resources deletion
• not making progress deleting resources; giving up
• failed to extract storage availability zones: %v
• Volume.RenderOpenstack: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/a23887f81cd34ef4. Report an issue: GitHub.