hashicorp/nomad · error

plugin %q for volume %q not found

Error message

plugin %q for volume %q not found

What it means

Returned by CSIVolumeEndpoint.Delete when the volume record exists but its referenced CSI plugin (vol.PluginID) is no longer registered in the agent's plugin registry. Without the plugin handle Nomad cannot send the ControllerDeleteVolume RPC to the storage provider, so it refuses to delete rather than silently orphaning the backend volume.

Source

Thrown at nomad/csi_endpoint.go:1472

	}

	if len(args.VolumeIDs) == 0 {
		return fmt.Errorf("missing volume IDs")
	}

	for _, volID := range args.VolumeIDs {

		plugin, vol, err := v.volAndPluginLookup(args.Namespace, volID)
		if err != nil {
			if err == fmt.Errorf("volume not found: %s", volID) {
				v.logger.Warn("volume %q to be deleted was already deregistered")
				continue
			} else {
				return err
			}
		}
		if plugin == nil {
			return fmt.Errorf("plugin %q for volume %q not found", vol.PluginID, volID)
		}

		// NOTE: deleting the volume in the external storage provider can't be
		// made atomic with deregistration. We can't delete a volume that's
		// not registered because we need to be able to lookup its plugin.
		err = v.deleteVolume(vol, plugin, args.Secrets)
		if err != nil {
			return err
		}
	}

	deregArgs := &structs.CSIVolumeDeregisterRequest{
		VolumeIDs:    args.VolumeIDs,
		WriteRequest: args.WriteRequest,
	}
	_, index, err := v.srv.raftApply(structs.CSIVolumeDeregisterRequestType, deregArgs)
	if err != nil {
		v.logger.Error("csi raft apply failed", "error", err, "method", "deregister")

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Restore/redeploy the CSI plugin job (same plugin ID) so the controller is registered, then retry the volume delete.
2. Use `nomad plugin status` to confirm the plugin is healthy and `nomad volume status <id>` to confirm vol.PluginID matches the running plugin.
3. If the plugin is permanently gone, deregister the volume instead of deleting it, and delete the volume manually in the storage provider.
4. Prevent by deleting volumes before stopping the CSI plugin job in teardown runbooks.

Example fix

# before: plugin already stopped, delete fails
nomad volume delete web-data
# after: re-run the csi plugin job first
nomad job run csi-plugin.nomad && nomad volume delete web-data

v, _, err := client.CSIVolumes().Get(ns, id, nil)
if err != nil { return err }
p, _, err := client.CSIPlugins().Get(v.PluginID, nil)
if err != nil || p.ControllersHealthy == 0 {
    return fmt.Errorf("plugin %s for volume %s is not registered/healthy; redeploy CSI plugin first", v.PluginID, id)
}

func isPluginMissing(err error) bool {
    return err != nil && strings.Contains(err.Error(), "not found") && strings.Contains(err.Error(), "plugin")
}

err := client.CSIVolumes().Delete(ns, []string{id}, secrets)
if err != nil && strings.Contains(err.Error(), "plugin") && strings.Contains(err.Error(), "not found") {
    // redeploy the CSI plugin job, then retry the delete
}

When it happens

Trigger: Calling `nomad volume delete` after the CSI plugin job was stopped/failed, the controller node was drained/removed, or the plugin deregistered due to health-check failure while the volume record still exists in state.

Common situations: Cleanup scripts run after tearing down the plugin deployment; plugin job crashed and Nomad GC'd its registration; upgrading/migrating storage drivers leaving stale volume registrations; wrong cluster (volume from a different cluster's plugin).

Related errors

• CSI Plugin loaded incorrectly
• CSI.ControllerListVolumes: plugin returned an invalid entry
• CSI.NodeDetachVolume: %v
• controller attach volume: %v
• controller validate volume: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/a82977e4c24c04c6. Report an issue: GitHub.