hashicorp/nomad · error

could not claim volume %s: %w

Error message

could not claim volume %s: %w

What it means

During Prerun, claimVolumes sends a CSIVolumeClaimRequest to the Nomad server via claimWithRetry; the server marks the volume claimed by this allocation and runs controller publish. Any error from that RPC (wrapped with %w) aborts allocation startup. This is the error boundary for server-side claim failures such as volume-not-found, claim conflicts, or RPC/network failures.

Source

Thrown at client/allocrunner/csi_hook.go:326

		req := &structs.CSIVolumeClaimRequest{
			VolumeID:       result.stub.VolumeID,
			AllocationID:   c.alloc.ID,
			NodeID:         c.alloc.NodeID,
			ExternalNodeID: result.stub.ExternalNodeID,
			Claim:          claimType,
			AccessMode:     request.AccessMode,
			AttachmentMode: request.AttachmentMode,
			WriteRequest: structs.WriteRequest{
				Region:    c.alloc.Job.Region,
				Namespace: c.alloc.Job.Namespace,
				AuthToken: c.nodeSecret,
			},
		}

		resp, err := c.claimWithRetry(req)
		if err != nil {
			return fmt.Errorf("could not claim volume %s: %w", req.VolumeID, err)
		}
		if resp.Volume == nil {
			return fmt.Errorf("Unexpected nil volume returned for ID: %v", request.Source)
		}

		result.volume = resp.Volume

		// populate data we'll write later to disk
		result.stub.VolumeID = resp.Volume.ID
		result.stub.VolumeNamespace = resp.Volume.Namespace
		result.stub.VolumeExternalID = resp.Volume.RemoteID()
		result.stub.PluginID = resp.Volume.PluginID
		result.publishContext = resp.PublishContext
	}

	return nil
}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Read the wrapped (%w) inner error to identify the root cause — volume missing, claim conflict, or RPC failure.
2. Verify the volume still exists with `nomad volume status <volume_id>` and re-register it if it was GC'd or deregistered.
3. Check the CSI plugin/controller health (`nomad plugin status`) and fix the plugin or its external storage backend.
4. Check access-mode compatibility: a single-writer volume cannot be claimed by multiple allocations; reduce claim count or change access_mode.
5. Retry allocation placement once server/plugin connectivity is restored (Nomad's claimWithRetry already retries transient errors).

// pre-check the volume exists and is claimable
vol, err := client.Volumes().Info(volumeID, nil)
if err != nil || vol == nil {
    return fmt.Errorf("volume %s not registered or inaccessible", volumeID)
}

func volumeClaimed(vol *api.CSIVolume) bool { return vol != nil && vol.ID != "" }

err := allocRunner.Prerun()
var claimErr *fmt.wrapError // inspect via errors.Unwrap chain
if err != nil && strings.Contains(err.Error(), "could not claim volume") {
    inner := errors.Unwrap(err)
    log.Printf("claim failed: %v; check volume status and plugin health", inner)
    // reschedule after fixing volume/plugin state
}

When it happens

Trigger: claimWithRetry returns an error for the volume identified by req.VolumeID: the volume was deregistered, the server rejected the claim (e.g. incompatible access mode, max claims reached, garbage-collected volume), the CSI plugin/controller is unhealthy, or the client cannot reach the server.

Common situations: Volume was GC'd or deregistered between job submission and allocation; too many readers/writers already claim the volume for its access mode; CSI controller plugin is down; node lost registration with the external storage system; server RPC errors/timeouts.

Related errors

• restoring mounts: %w
• claiming volumes: %w
• no task supports CSI
• Unexpected nil volume returned for ID: %v
• No CSI Mount Point found for volume: %s

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/1a5e3ece1942bf49. Report an issue: GitHub.