What causes "failed to delete volume: %s, err: %s" in kubernetes/kops?

Check the DigitalOcean API token and volume delete permissions

failed to delete volume: %s, err: %s

Error message

failed to delete volume: %s, err: %s

What it means

deleteVolume wraps the godo VolumeService.DeleteVolume failure after detaching all droplets. It fires when DO refuses to delete the volume — commonly because a detach is still in progress or the volume is still attached; detached-already (404) cases are skipped earlier in the function.

Source

Thrown at pkg/resources/digitalocean/resources.go:299

	volume := t.Obj.(godo.Volume)
	for _, dropletID := range volume.DropletIDs {
		action, resp, err := c.VolumeActionService().DetachByDropletID(context.TODO(), volume.ID, dropletID)
		if err != nil {
			if resp != nil && resp.StatusCode == http.StatusNotFound {
				// Volume is already detached, nothing to do.
				continue
			}
			return fmt.Errorf("failed to detach volume %s: %s", volume.ID, err)
		}

		if err := waitForDetach(c, action); err != nil {
			return fmt.Errorf("error while waiting for volume %s to detach: %s", volume.ID, err)
		}
	}

	_, err := c.VolumeService().DeleteVolume(context.TODO(), t.ID)
	if err != nil {
		return fmt.Errorf("failed to delete volume: %s, err: %s", t.ID, err)
	}

	return nil
}

func deleteRecord(cloud fi.Cloud, domain string, t *resources.Resource) error {
	c := cloud.(do.DOCloud)
	record := t.Obj.(godo.DomainRecord)

	_, err := c.DomainService().DeleteRecord(context.TODO(), domain, record.ID)
	if err != nil {
		return fmt.Errorf("failed to delete record for domain %s: %d", domain, record.ID)
	}

	return nil
}

func deleteLoadBalancer(cloud fi.Cloud, t *resources.Resource) error {

View on GitHub (pinned to 4c8573c808)

Solutions

Check the DigitalOcean API token and volume delete permissions
Verify the volume is fully detached, then retry
404 means the volume was already deleted — re-run discovery

Example fix

// before
# delete refused: volume still marked attached
// after
# retry after detach propagation, or:
doctl compute volume delete <volume-id>
kops delete cluster --cloud digitalocean ...

Defensive patterns

Strategy: retry

Validate before calling

vol, resp, err := c.VolumeService().GetVolume(context.TODO(), t.ID)
if resp != nil && resp.StatusCode == http.StatusNotFound {
	return nil // already deleted
}
if err != nil { return err }
_ = vol

Try / catch

if _, err := c.VolumeService().DeleteVolume(context.TODO(), t.ID); err != nil {
	var gerr *godo.ErrorResponse
	if errors.As(err, &gerr) && gerr.Response.StatusCode == http.StatusNotFound {
		return nil // idempotent
	}
	return fmt.Errorf("failed to delete volume: %s, err: %w", t.ID, err)
}

Prevention

Treat 404 as idempotent success in delete paths
Add a small delay after detach before delete to let attach-state propagate
Ensure token has volume delete scope

When it happens

Trigger: DeleteVolume returns 404 (already deleted), 403 (scope), or an API/network error; deletion can also fail if DO still considers the volume attached.

Common situations: Volume deleted manually before kops delete; detach reported complete but DO hasn't propagated; token lacking volume delete scope; transient API failure.

Related errors

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/47df9d3b195fbc83. Report an issue: GitHub.