hashicorp/nomad · error

controller list snapshots: %v

Error message

controller list snapshots: %v

What it means

Nomad's ClientCSI.ControllerListSnapshots forwards a CSI.ControllerListSnapshots RPC to a controller plugin via sendCSIControllerRPC, wrapping failures as 'controller list snapshots: %v'. Nomad could not enumerate snapshots from the storage backend through the plugin.

Source

Thrown at nomad/client_csi_endpoint.go:168

		"ClientCSI.ControllerDeleteSnapshot",
		structs.RateMetricWrite,
		args, reply)
	if err != nil {
		return fmt.Errorf("controller delete snapshot: %v", err)
	}
	return nil
}

func (a *ClientCSI) ControllerListSnapshots(args *cstructs.ClientCSIControllerListSnapshotsRequest, reply *cstructs.ClientCSIControllerListSnapshotsResponse) error {
	defer metrics.MeasureSince([]string{"nomad", "client_csi_controller", "list_snapshots"}, time.Now())

	err := a.sendCSIControllerRPC(args.PluginID,
		"CSI.ControllerListSnapshots",
		"ClientCSI.ControllerListSnapshots",
		structs.RateMetricList,
		args, reply)
	if err != nil {
		return fmt.Errorf("controller list snapshots: %v", err)
	}
	return nil
}

func (a *ClientCSI) sendCSIControllerRPC(pluginID, method, fwdMethod, op string, args cstructs.CSIControllerRequest, reply any) error {

	// client requests aren't RequestWithIdentity, so we use a placeholder here
	// to populate the identity data for metrics
	identityReq := &structs.GenericRequest{}

	aclObj, err := a.srv.AuthenticateServerOnly(a.ctx, identityReq)
	a.srv.MeasureRPCRate("client_csi", op, identityReq)

	if err != nil || !aclObj.AllowServerOp() {
		return structs.ErrPermissionDenied
	}

	clientIDs, err := a.clientIDsForController(pluginID)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check `nomad plugin status <pluginID>` for a healthy controller and start/restart its job.
2. Confirm the driver supports listing snapshots (CSI LIST_SNAPSHOTS capability).
3. Verify backend credentials/permissions in the plugin job's secrets.
4. Read the wrapped root cause in server logs and fix connectivity or plugin errors.

Example fix

# before: listing snapshots while controller is down
nomad volume snapshots -plugin aws-ebs-controller   # fails
# after
nomad job run aws-ebs-csi-controller.nomad.hcl
nomad plugin status aws-ebs-controller && nomad volume snapshots -plugin aws-ebs-controller

// ensure driver supports listing and controller is healthy
if p.ControllersHealthy == 0 { return fmt.Errorf("no healthy controller for listing snapshots") }

func supportsListSnapshots(caps []string) bool {
    for _, c := range caps { if c == "LIST_SNAPSHOTS" { return true } }
    return false
}

err := csi.ControllerListSnapshots(args, reply)
if err != nil {
    if strings.Contains(err.Error(), "controller list snapshots:") {
        // retry after checking plugin health; cap retries
    }
    return err
}

When it happens

Trigger: Calling ControllerListSnapshots (e.g. `nomad volume snapshots`) when the controller plugin has no healthy instances, the client connection is broken, or the driver's ListSnapshots call fails (pagination/backend error).

Common situations: Plugin not running or still starting; backend credentials revoked so the driver can't list; driver does not support LIST_SNAPSHOTS capability; network partition.

Related errors

• controller create snapshot: %v
• controller delete snapshot: %v
• volume snapshot ID cannot be updated
• controller list volumes: %v
• %s error: %w

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/f4b300a9c1395694. Report an issue: GitHub.