hashicorp/nomad · error

could not query host volume: %w

Error message

could not query host volume: %w

What it means

Delete looks up the volume via snap.HostVolumeByID; if the state store query itself returns an error (rather than a nil volume), the server wraps it with 'could not query host volume'. This indicates an internal state-store failure, not a missing volume — the distinct 'no such volume' error handles the nil case.

Source

Thrown at nomad/host_volume_endpoint.go:696

	if !allowVolume(aclObj, args.RequestNamespace()) {
		return structs.ErrPermissionDenied
	}

	if args.VolumeID == "" {
		return fmt.Errorf("missing volume ID to delete")
	}

	snap, err := v.srv.State().Snapshot()
	if err != nil {
		return err
	}

	ns := args.RequestNamespace()
	id := args.VolumeID

	vol, err := snap.HostVolumeByID(nil, ns, id, true)
	if err != nil {
		return fmt.Errorf("could not query host volume: %w", err)
	}
	if vol == nil {
		return fmt.Errorf("no such volume: %s", id)
	}
	if len(vol.Allocations) > 0 {
		allocIDs := helper.ConvertSlice(vol.Allocations,
			func(a *structs.AllocListStub) string { return a.ID })
		return fmt.Errorf("volume %s in use by allocations: %v", id, allocIDs)
	}

	// serialize client RPC and raft write per volume ID
	index, err := v.serializeCall(vol.ID, "delete", func() (uint64, error) {
		if err := v.deleteVolume(vol); err != nil {
			if structs.IsErrUnknownNode(err) || structs.IsErrNoNodeConn(err) {
				if !args.Force {
					return 0, fmt.Errorf(
						"volume cannot be removed from unknown node without force=true")
				}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Retry the delete — transient state errors often resolve on the next attempt or after leadership stabilizes.
2. Check server logs around the error for the underlying state-store cause.
3. If persistent, inspect server health/disk and consider restarting the unhealthy server or restoring from a known-good state.

Example fix

// client-side resilience
err := client.HostVolumes().Delete(ns, id, nil)
if err != nil && strings.Contains(err.Error(), "could not query host volume") {
    time.Sleep(2 * time.Second)
    err = client.HostVolumes().Delete(ns, id, nil) // retry transient state error
}

err := client.HostVolumes().Delete(ns, id, nil)
if err != nil && strings.Contains(err.Error(), "could not query host volume") {
    // transient state-store error; bounded retry
    for i := 0; i < 3; i++ {
        time.Sleep(time.Duration(1<<i) * time.Second)
        if err = client.HostVolumes().Delete(ns, id, nil); err == nil || !strings.Contains(err.Error(), "could not query host volume") { break }
    }
}

When it happens

Trigger: A memdb/state-store error while reading HostVolumeByID during Delete — corrupted state, snapshot failure, or low-level store errors; not caused by a nonexistent volume (that yields 'no such volume' instead).

Common situations: State store instability after crashes; disk I/O issues on the server; rare internal errors during leader transitions.

Related errors

• validating volume %q against state failed: %v
• could not place volume %q: %w
• error looking up job version by tag: %v
• failed to query node pool: %v
• error parsing: root should be an object

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/c68066e089a74579. Report an issue: GitHub.