hashicorp/nomad · error

controller publish: %v

Error message

controller publish: %v

What it means

For a new volume claim, Claim calls controllerPublishVolume to have the CSI controller publish the volume to the node. Any error from that controller RPC is wrapped as 'controller publish: %v', so the root cause is in the nested error (plugin errors, node issues, RPC failures).

Source

Thrown at nomad/csi_endpoint.go:505

		if alloc == nil {
			return fmt.Errorf("%s: %s",
				structs.ErrUnknownAllocationPrefix, args.AllocationID)
		}
		args.NodeID = alloc.NodeID
	}

	_, index, err := v.srv.raftApply(structs.CSIVolumeClaimRequestType, args)
	if err != nil {
		v.logger.Error("csi raft apply failed", "error", err, "method", "claim")
		return err
	}

	if isNewClaim {
		// if this is a new claim, add a Volume and PublishContext from the
		// controller (if any) to the reply
		err = v.controllerPublishVolume(args, reply)
		if err != nil {
			return fmt.Errorf("controller publish: %v", err)
		}
	}

	reply.Index = index
	v.srv.setQueryMeta(&reply.QueryMeta)
	return nil
}

func csiVolumeMountOptions(c *structs.CSIMountOptions) *cstructs.CSIVolumeMountOptions {
	if c == nil {
		return nil
	}

	return &cstructs.CSIVolumeMountOptions{
		Filesystem: c.FSType,
		MountFlags: c.MountFlags,
	}
}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Read the wrapped inner error to find the controller failure cause and fix that (e.g. restart controller job, fix storage backend)
2. Verify plugin/controller health with nomad plugin status <plugin>
3. Check that the volume exists at the provider and the target node is healthy, then retry the claim

p, _ := client.CSIPlugins().Get(pluginID, nil)
if p == nil || p.ControllersHealthy < 1 {
    return errors.New("controller unavailable for publish")
}

if err := claim(); err != nil {
    if wrapped := errors.Unwrap(err); wrapped != nil {
        // inspect controller publish cause and retry with backoff
    }
}

When it happens

Trigger: A new claim (not GC) on a volume whose plugin has a controller, and the controller publish RPC fails — e.g. controller unreachable, volume not yet created at the provider, or node not viable.

Common situations: CSI controller job down or not scheduled; external storage backend errors (volume doesn't exist, capacity exceeded); network partition between Nomad client and controller; stale plugin registration pointing to a dead node.

Related errors

• cannot upload management tokens
• CSI.ControllerValidateVolume: %w: %v (wraps ErrCSIClientRPCR
• failed to find instances of controller plugin %q
• plugin does not support snapshot
• could not delete %q: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/4e44ca2480f38812. Report an issue: GitHub.