What causes "volume not found: %s" in hashicorp/nomad?

List volumes to confirm the ID and namespace: `nomad volume status -namespace ` or GET /v1/csi/volumes.

volume not found: %s

Error message

volume not found: %s

What it means

volAndPluginLookup cannot find the requested CSI volume in Nomad's state store and returns "volume not found: <id>". This helper backs controllerPublishVolume and CSIVolume.Delete, so any claim or delete against a volume ID that is not registered under the given namespace produces this error. The volume ID and namespace must both match a registered volume.

Source

Thrown at nomad/csi_endpoint.go:608

	})
	if err != nil {
		if strings.Contains(err.Error(), "FailedPrecondition") {
			return fmt.Errorf("%v: %v", structs.ErrCSIClientRPCRetryable, err)
		}
		return err
	}
	resp.PublishContext = cResp.PublishContext
	return nil
}

func (v *CSIVolume) volAndPluginLookup(namespace, volID string) (*structs.CSIPlugin, *structs.CSIVolume, error) {
	state := v.srv.fsm.State()
	vol, err := state.CSIVolumeByID(nil, namespace, volID)
	if err != nil {
		return nil, nil, err
	}
	if vol == nil {
		return nil, nil, fmt.Errorf("volume not found: %s", volID)
	}
	if !vol.ControllerRequired {
		return nil, vol, nil
	}

	// note: we do this same lookup in CSIVolumeByID but then throw
	// away the pointer to the plugin rather than attaching it to
	// the volume so we have to do it again here.
	plug, err := state.CSIPluginByID(nil, vol.PluginID)
	if err != nil {
		return nil, nil, err
	}
	if plug == nil {
		return nil, nil, fmt.Errorf("plugin not found: %s", vol.PluginID)
	}
	return plug, vol, nil
}

View on GitHub (pinned to 482b49bf1a)

Solutions

List volumes to confirm the ID and namespace: `nomad volume status -namespace <ns>` or GET /v1/csi/volumes.
Re-register the volume: `nomad volume register <volume-file>` if it was deregistered.
Fix the job's volume source / claim request to use the exact registered volume ID and correct namespace.
Check ACL policies — a policy scoped to the wrong namespace will hide the volume from the caller.

Example fix

# before (job file)
volume "db" { type = "csi" source = "db-vol" }  # no volume named db-vol
# after
volume "db" { type = "csi" source = "ebs-db-vol" }  # matches `nomad volume status ebs-db-vol`

Defensive patterns

Strategy: validation

Validate before calling

vols, _, err := client.CSIVolumes().List(&capi.VolumeListOptions{Namespace: ns})
if err != nil { return err }
found := false
for _, v := range vols { if v.ID == volID { found = true } }
if !found { return fmt.Errorf("volume %s not registered in namespace %s", volID, ns) }

Try / catch

if err != nil && strings.Contains(err.Error(), "volume not found") {
    // register the volume or fix the job's volume.source before retrying
}

Prevention

Keep job `volume.source` values identical to registered CSI volume IDs.
Don't deregister volumes while allocations still claim them.
Verify namespace in ACL tokens/requests matches the volume's namespace.

When it happens

Trigger: Raised when state.CSIVolumeByID(nil, namespace, volID) returns nil for the (namespace, volID) pair — the volume was deregistered (`nomad volume deregister`/`volume delete`), never registered, registered in a different namespace, or the ID was mistyped.

Common situations: Jobs reference volumes in host/CSI `volume` blocks whose registered ID doesn't match the `volume.source`; volumes deregistered while allocations still request them; ACL/namespace confusion where the caller's namespace differs from where the volume lives (e.g. `default` vs a team namespace).

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/6fb71be0be29fc91. Report an issue: GitHub.