{"record":{"id":"8192d828b9fe2099","repo":"hashicorp/nomad","slug":"controller-delete-volume-v","errorCode":null,"errorMessage":"controller delete volume: %v","messagePattern":"controller delete volume: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"nomad/client_csi_endpoint.go","lineNumber":112,"sourceCode":"\t\t\"ClientCSI.ControllerExpandVolume\",\n\t\tstructs.RateMetricWrite,\n\t\targs, reply)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"controller expand volume: %v\", err)\n\t}\n\treturn nil\n}\n\nfunc (a *ClientCSI) ControllerDeleteVolume(args *cstructs.ClientCSIControllerDeleteVolumeRequest, reply *cstructs.ClientCSIControllerDeleteVolumeResponse) error {\n\tdefer metrics.MeasureSince([]string{\"nomad\", \"client_csi_controller\", \"delete_volume\"}, time.Now())\n\n\terr := a.sendCSIControllerRPC(args.PluginID,\n\t\t\"CSI.ControllerDeleteVolume\",\n\t\t\"ClientCSI.ControllerDeleteVolume\",\n\t\tstructs.RateMetricWrite,\n\t\targs, reply)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"controller delete volume: %v\", err)\n\t}\n\treturn nil\n}\n\nfunc (a *ClientCSI) ControllerListVolumes(args *cstructs.ClientCSIControllerListVolumesRequest, reply *cstructs.ClientCSIControllerListVolumesResponse) error {\n\tdefer metrics.MeasureSince([]string{\"nomad\", \"client_csi_controller\", \"list_volumes\"}, time.Now())\n\n\terr := a.sendCSIControllerRPC(args.PluginID,\n\t\t\"CSI.ControllerListVolumes\",\n\t\t\"ClientCSI.ControllerListVolumes\",\n\t\tstructs.RateMetricList,\n\t\targs, reply)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"controller list volumes: %v\", err)\n\t}\n\treturn nil\n}\n","sourceCodeStart":94,"sourceCodeEnd":130,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/client_csi_endpoint.go#L94-L130","documentation":"ClientCSI.ControllerDeleteVolume forwards a CSI ControllerDeleteVolume RPC to the controller plugin and wraps failures as \"controller delete volume: <err>\". It runs when a dynamically provisioned volume is deregistered with purge, asking the controller to delete the backing storage object.","triggerScenarios":"`nomad volume deregister -purge` (or deregistration of a dynamic volume) when the controller plugin is unreachable or the backend refuses deletion (volume still attached, snapshots exist, volume is external/not created by the plugin).","commonSituations":"Claims still outstanding so the backend keeps the volume; attempting to purge an externally pre-provisioned volume the controller won't delete; controller plugin down; storage-side dependency (snapshot, clone).","solutions":["Release all claims first: stop jobs using the volume, then `nomad volume detach` / `nomad volume claim -delete` until no claims remain, then retry purge.","If the volume is external (not dynamically created), deregister WITHOUT -purge; deletion is not applicable.","Restart/repair the controller plugin and check its logs for the backend deletion error; resolve storage-side dependencies (snapshots, attachments) then retry."],"exampleFix":"# before: purge fails because claims remain\nnomad volume deregister -purge ebs-vol\n# after\nnomad volume detach ebs-vol <node-id>\nnomad volume claim -delete ebs-vol <node-id> <alloc-id>\nnomad volume deregister -purge ebs-vol","handlingStrategy":"validation","validationCode":"const vol = await nomad.volume(volumeID)\nconst outstanding = await nomad.volumeClaims(volumeID)\nif (vol.externalId && !vol.dynamicallyCreated) {\n  throw new Error('external volume: use deregister without -purge; the controller will not delete it')\n}\nif (outstanding.length > 0) {\n  throw new Error(`release ${outstanding.length} claim(s) before purging`)\n}","typeGuard":"const canPurge = (vol, claims) => vol.dynamicallyCreated && claims.length === 0","tryCatchPattern":"try { await deregisterVolume(id, { purge: true }) }\ncatch (e) {\n  if (String(e).startsWith('controller delete volume')) {\n    await releaseAllClaims(id)\n    return deregisterVolume(id, { purge: true }) // retry once after cleanup\n  }\n  throw e\n}","preventionTips":["Always release all claims (detach + claim -delete) before purging.","Only use -purge on dynamically provisioned volumes.","Delete storage-side dependencies (snapshots, clones) beforehand.","Verify controller plugin health as a prerequisite for deletion workflows."],"tags":["nomad","csi","storage","deletion","plugin"],"backgroundTag":"csi-controller-rpc-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}