What causes "failed to delete or recycle" in derailed/k9s?

Run kubectl describe pv and read Events to find why reclaim failed

failed to delete or recycle

Error message

failed to delete or recycle

What it means

PersistentVolume.diagnose is the health check behind the VALID column: when a PV's status.phase equals v1.VolumeFailed it returns 'failed to delete or recycle', which AsStatus renders as row status text. It means the reclaim policy (Delete or Recycle) attempted to free the backing storage and the storage layer failed. The error describes cluster state, not a bug in the caller, and it never propagates out of Render.

Source

Thrown at internal/render/pv.go:131

		size.String(),
		accessMode(pv.Spec.AccessModes),
		string(pv.Spec.PersistentVolumeReclaimPolicy),
		string(phase),
		claim,
		class,
		pv.Status.Reason,
		p.volumeMode(pv.Spec.VolumeMode),
		mapToStr(pv.Labels),
		AsStatus(p.diagnose(phase)),
		ToAge(pv.GetCreationTimestamp()),
	}

	return nil
}

func (PersistentVolume) diagnose(phase v1.PersistentVolumePhase) error {
	if phase == v1.VolumeFailed {
		return fmt.Errorf("failed to delete or recycle")
	}

	return nil
}

func (PersistentVolume) volumeMode(m *v1.PersistentVolumeMode) string {
	if m == nil {
		return MissingValue
	}

	return string(*m)
}

// ----------------------------------------------------------------------------
// Helpers...

func accessMode(aa []v1.PersistentVolumeAccessMode) string {
	dd := accessDedup(aa)

View on GitHub (pinned to 2d3ccc6ba2)

Solutions

Run kubectl describe pv <name> and read Events to find why reclaim failed
Fix the CSI driver / cloud credentials so deletion succeeds and let the controller retry
Protect data first: kubectl patch pv <name> -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' and clean up manually
If the PV is truly orphaned, remove stale finalizers (e.g. storage.kubernetes.io/pv-protection) only after verifying no data is at risk

Example fix

# before: PV stuck Failed with reclaimPolicy Delete
kubectl get pv pvc-123 -o jsonpath='{.status.phase} {..persistentVolumeReclaimPolicy}'
# Failed Delete
# after: keep the data, stop the failed reclaim loop
kubectl patch pv pvc-123 -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'

Defensive patterns

Strategy: validation

Validate before calling

# surface PVs that will trip the diagnostic before opening the view
kubectl get pv -o json | jq '.items[] | select(.status.phase=="Failed") | {name:.metadata.name, policy:.spec.persistentVolumeReclaimPolicy}'

Prevention

Set persistentVolumeReclaimPolicy to Retain for volumes holding data you cannot lose
Keep the CSI driver and cloud credentials healthy so Delete reclaim succeeds
Watch PV events: a Failed phase is a storage-layer problem, not a UI error
Treat the VALID column text as a signal to investigate with kubectl describe, not an exception to catch

When it happens

Trigger: A PV with spec.persistentVolumeReclaimPolicy Delete or Recycle whose deletion at the CSI driver or cloud provider failed (driver errors, missing permissions, backing cloud already deleted); browsing the PV view in k9s while such a PV exists.

Common situations: Decommissioning a cluster while PVs still use Delete policy; broken CSI driver or cloud credentials; provisioning backends that do not implement deletion; orphaned PVs after the storage backend was removed.

Related errors

AI-assisted analysis of derailed/k9s@2d3ccc6ba2 (2026-08-15). Data as JSON: /api/errors/2b587177b7651d02. Report an issue: GitHub.