hashicorp/nomad · error

CSI.ControllerListSnapshots: %v

Error message

CSI.ControllerListSnapshots: %v

What it means

This wraps the error from the plugin's ControllerListSnapshots gRPC call, which is executed with a per-attempt timeout (CSIPluginRequestTimeout) and up to 3 attempts with exponential backoff for retryable codes. Any error still returned after retries is surfaced here with the underlying gRPC/CSI status embedded.

Source

Thrown at client/csi_endpoint.go:471

		// should retry with another controller instance
		return fmt.Errorf("CSI.ControllerListSnapshots: %w: %v",
			nstructs.ErrCSIClientRPCRetryable, err)
	}
	defer plugin.Close()

	csiReq := req.ToCSIRequest()

	ctx, cancelFn := c.requestContext()
	defer cancelFn()

	// CSI ControllerListSnapshots errors for timeout, codes.Unavailable and
	// codes.ResourceExhausted are retried; all other errors are fatal.
	cresp, err := plugin.ControllerListSnapshots(ctx, csiReq,
		grpc_retry.WithPerRetryTimeout(CSIPluginRequestTimeout),
		grpc_retry.WithMax(3),
		grpc_retry.WithBackoff(grpc_retry.BackoffExponential(100*time.Millisecond)))
	if err != nil {
		return fmt.Errorf("CSI.ControllerListSnapshots: %v", err)
	}

	resp.NextToken = cresp.NextToken
	resp.Entries = []*nstructs.CSISnapshot{}

	for _, entry := range cresp.Entries {
		if entry.Snapshot == nil {
			return fmt.Errorf("CSI.ControllerListSnapshot: plugin returned an invalid entry")
		}
		snap := &nstructs.CSISnapshot{
			ID:                     entry.Snapshot.ID,
			ExternalSourceVolumeID: entry.Snapshot.SourceVolumeID,
			SizeBytes:              entry.Snapshot.SizeBytes,
			CreateTime:             entry.Snapshot.CreateTime,
			IsReady:                entry.Snapshot.IsReady,
			PluginID:               req.PluginID,
		}
		resp.Entries = append(resp.Entries, snap)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Read the wrapped status code; for ResourceExhausted, reduce list frequency or page size (Secrets/MaxEntries, NextToken)
2. Retry the list later if the backend was throttling or briefly unavailable
3. Check controller plugin logs and backend health for outages
4. Use next_token pagination to make listing calls smaller and less likely to time out

Example fix

// before: one giant unpaginated list
req := &csi.ListSnapshotsRequest{}
// after: paginate
req := &csi.ListSnapshotsRequest{MaxEntries: 100, StartingToken: nextToken}

// Reduce result size before listing to avoid backend throttling/timeout:
req := &structs.ClientCSIControllerListSnapshotsRequest{
    PluginID: pluginID,
    NextToken: nextToken, // paginate
    MaxEntries: 100,
}

// Inspect wrapped status; retry only transient codes
err := c.ControllerListSnapshots(req, resp)
if err != nil {
    if st, ok := status.FromError(errors.Unwrap(err)); ok &&
        (st.Code() == codes.Unavailable || st.Code() == codes.ResourceExhausted) {
        // back off longer, then retry
    }
}

When it happens

Trigger: ListSnapshots against the plugin fails after internal retries: deadline exceeded, persistent Unavailable, ResourceExhausted (e.g. backend throttling list calls), or fatal CSI codes from the backend API.

Common situations: Cloud API rate limiting when listing many snapshots; slow storage backends exceeding the per-attempt timeout; plugin/backend connectivity issues; pagination requests against an unhealthy backend.

Related errors

• CSI.ControllerDeleteSnapshot: %v
• CSI.ControllerListSnapshot: plugin returned an invalid entry
• invalid starting token %q: %v
• snapshot %q already exists but is incompatible with volume I
• snapshot %q is already pending: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/bc093d4780ee9576. Report an issue: GitHub.