hashicorp/nomad · error

CSI.ControllerListVolumes: plugin returned an invalid entry

Error message

CSI.ControllerListVolumes: plugin returned an invalid entry

What it means

CSI.ControllerListVolumes wraps the server RPC ControllerListVolumes. After the plugin responds, every entry in cresp.Entries is expected to carry a non-nil Volume. If a plugin returns an entry with a nil Volume (a malformed ListVolumesResponse), Nomad rejects the whole response instead of producing a stub with empty fields, so the caller gets this 'plugin returned an invalid entry' error. It signals a bug or non-conformance in the CSI plugin, not in Nomad's configuration.

Source

Thrown at client/csi_endpoint.go:344

	ctx, cancelFn := c.requestContext()
	defer cancelFn()

	// CSI ControllerListVolumes errors for timeout, codes.Unavailable and
	// codes.ResourceExhausted are retried; all other errors are fatal.
	cresp, err := plugin.ControllerListVolumes(ctx, csiReq,
		grpc_retry.WithPerRetryTimeout(CSIPluginRequestTimeout),
		grpc_retry.WithMax(3),
		grpc_retry.WithBackoff(grpc_retry.BackoffExponential(100*time.Millisecond)))
	if err != nil {
		return fmt.Errorf("CSI.ControllerListVolumes: %v", err)
	}

	resp.NextToken = cresp.NextToken
	resp.Entries = []*nstructs.CSIVolumeExternalStub{}

	for _, entry := range cresp.Entries {
		if entry.Volume == nil {
			return fmt.Errorf("CSI.ControllerListVolumes: plugin returned an invalid entry")
		}
		vol := &nstructs.CSIVolumeExternalStub{
			ExternalID:    entry.Volume.ExternalVolumeID,
			CapacityBytes: entry.Volume.CapacityBytes,
			VolumeContext: entry.Volume.VolumeContext,
			CloneID:       entry.Volume.ContentSource.CloneID,
			SnapshotID:    entry.Volume.ContentSource.SnapshotID,
		}
		if entry.Status != nil {
			vol.PublishedExternalNodeIDs = entry.Status.PublishedNodeIds
			vol.IsAbnormal = entry.Status.VolumeCondition.Abnormal
			if entry.Status.VolumeCondition != nil {
				vol.Status = entry.Status.VolumeCondition.Message
			}
		}
		resp.Entries = append(resp.Entries, vol)
		if req.MaxEntries != 0 && int32(len(resp.Entries)) == req.MaxEntries {
			break

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Report the bug to the CSI plugin vendor/author; the plugin must never emit entries with nil Volume
2. Upgrade the CSI plugin to the latest version where list-response handling may be fixed
3. Deduce the offending plugin from the volume context and test its ListVolumes directly with csc controller list-volumes
4. As a workaround, narrow the listing (pagination/next_token or volume ID filters) to avoid the malformed entries

Example fix

// before (plugin side, pseudo-Go)
entries = append(entries, &csi.ListVolumesResponse_Entry{})
// after
entries = append(entries, &csi.ListVolumesResponse_Entry{
    Volume: &csi.Volume{VolumeId: id, CapacityBytes: cap},
})

// Before relying on list output, sanity-check via the plugin directly:
// nomad volume status  (external listing) and ensure the plugin version is current.
// There is no caller-side pre-check inside Nomad; treat nil entries as plugin bugs.
if entry.Volume == nil { /* plugin bug: report, upgrade plugin */ }

func isValidVolumeEntry(e *csi.ListVolumesResponse_Entry) bool {
    return e != nil && e.Volume != nil
}

// Go: check the returned error from ControllerListVolumes
if err := c.ControllerListVolumes(req, resp); err != nil {
    if strings.Contains(err.Error(), "plugin returned an invalid entry") {
        // fall back to another plugin version / report to vendor
    }
}

When it happens

Trigger: A CSI controller plugin's ListVolumes RPC returns a response whose Entries slice contains an element with a nil Volume field; the server RPC reached this client and iterated over that entry.

Common situations: Buggy or pre-spec CSI plugin builds that emit placeholder/partial list entries; plugins that paginate by appending empty marker entries; a plugin upgraded to a version that changed its list response shape; vendor SDKs that allow nil volumes in list results.

Related errors

• node plugin returned an error: %v
• CSI Plugin loaded incorrectly
• CSI.ControllerListSnapshot: plugin returned an invalid entry
• CSI.NodeDetachVolume: %v
• controller attach volume: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/9d2faba6180bf9bb. Report an issue: GitHub.