volume %q is already published on another node and does not

Error message

volume %q is already published on another node and does not have MULTI_NODE volume capability: %v

What it means

Raised in ControllerPublishVolume when the CSI controller returns gRPC FailedPrecondition. The volume (req.ExternalID) is already published (CONTROLLER_PUBLISH_VOLUME) on a different node, and the volume was not provisioned with MULTI_NODE (MULTI_NODE_READER_ONLY / MULTI_NODE_SINGLE_WRITER etc.) capability, so it cannot also be attached to req.NodeID. Nomad surfaces this so callers can distinguish an attach-conflict from other publish failures.

Source

Thrown at plugins/csi/client.go:328

	}

	pbrequest := req.ToCSIRepresentation()
	resp, err := c.controllerClient.ControllerPublishVolume(ctx, pbrequest, opts...)
	if err != nil {
		code := status.Code(err)
		switch code {
		case codes.NotFound:
			err = fmt.Errorf("volume %q or node %q could not be found: %v",
				req.ExternalID, req.NodeID, err)
		case codes.AlreadyExists:
			err = fmt.Errorf(
				"volume %q is already published at node %q but with capabilities or a read_only setting incompatible with this request: %v",
				req.ExternalID, req.NodeID, err)
		case codes.ResourceExhausted:
			err = fmt.Errorf("node %q has reached the maximum allowable number of attached volumes: %v",
				req.NodeID, err)
		case codes.FailedPrecondition:
			err = fmt.Errorf("volume %q is already published on another node and does not have MULTI_NODE volume capability: %v",
				req.ExternalID, err)
		case codes.Internal:
			err = fmt.Errorf("controller plugin returned an internal error, check the plugin allocation logs for more information: %v", err)
		}
		return nil, err
	}

	return &ControllerPublishVolumeResponse{
		PublishContext: maps.Clone(resp.PublishContext),
	}, nil
}

func (c *client) ControllerUnpublishVolume(ctx context.Context, req *ControllerUnpublishVolumeRequest, opts ...grpc.CallOption) (*ControllerUnpublishVolumeResponse, error) {
	if err := c.ensureConnected(ctx); err != nil {
		return nil, err
	}
	err := req.Validate()
	if err != nil {

View on GitHub (pinned to 482b49bf1a)

Solutions

Ensure the volume is unpublished from the old node first (verify the old allocation is stopped and ControllerUnpublishVolume succeeded), then retry the publish.
Check the CSI plugin/cluster logs and the Nomad volume checkpoint state for a stale publish on the previous node and force a detach via the storage backend console/CLI.
If the workload genuinely needs multi-node access, recreate the volume with MULTI_NODE capability (and ensure the plugin supports it).
Fix the root cause of missed unpublishes (e.g. node down → use the storage provider's force-detach), and re-run the job.

Example fix

// before: job pinned to a node that changed, old attachment lingers
//   volume "vol-123" is already published on another node ...
// after: detach first at the provider, then reschedule
# aws ec2 detach-volume --volume-id vol-123 --instance-id i-old --force
# nomad volume detach vol-123 i-old
# nomad job run app.nomad.hcl

Defensive patterns

Strategy: validation

Validate before calling

// Only one node can hold a single-node-writable volume.
// Before publishing, verify the volume has no active claim on another node:
vol, _, err := nomadClient.CSIVolumes().Get(nil, volumeID)
if err != nil { return err }
for _, r := range vol.Reads { if r.NodeID != targetNodeID { /* MULTI_NODE reader ok? */ } }
if len(vol.WriteAllocs) > 0 {
    for _, w := range vol.WriteAllocs {
        if w.NodeID != targetNodeID {
            return fmt.Errorf("volume %s still claimed on node %s; unpublish first", volumeID, w.NodeID)
        }
    }
}

Type guard

func isAttachConflict(err error) bool {
    return err != nil && strings.Contains(err.Error(),
        "already published on another node")
}

Try / catch

err := publishVolume(ctx, req)
if err != nil {
    if isAttachConflict(err) {
        // force-detach at the old node, then retry once
        _ = unpublishAtOldNode(ctx, volumeID, oldNodeID)
        err = publishVolume(ctx, req)
    }
    return err
}

Prevention

Ensure old allocations fully stop and unpublish completes before rescheduling stateful workloads.
Recreate volumes with MULTI_NODE capability if they must attach to several nodes.
Monitor CSI plugin/node crash events — skipped unpublishes are the usual cause of stale publishes.
Keep volume claim GC enabled so checkpointed claims get cleaned up.

When it happens

Trigger: c.ControllerPublishVolume() is called for a volume that is still attached to node A while the request targets node B, and the volume's declared capability set lacks MULTI_NODE access modes.

Common situations: A task was rescheduled from node A to node B but the old ControllerUnpublishVolume never completed; failover of a stateful workload to a standby node; a single-node RWO volume shared by groups on different clients; the plugin's unpublish was skipped due to a crashed client.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/3a0eaaffdfa980b5. Report an issue: GitHub.