hashicorp/nomad · info

CSI client error (ignorable)

Error message

CSI client error (ignorable)

What it means

ErrCSIClientRPCIgnorable marks a CSI client-side RPC error as safe to ignore — the operation can proceed or be retried without consequence. Nomad wraps or string-matches this sentinel because CSI plugin errors cross RPC boundaries and lose their Go type, so callers use strings.Contains to detect it. It signals transient/harmless plugin responses rather than real failures.

Source

Thrown at nomad/structs/errors.go:89

	// evaluate a requested filter expression. api.ResultPaginatorErrorContent
	// duplicates its message so the CLI can match it without importing structs.
	// Keep the two in sync.
	ErrResultPaginatorCreation = errors.New(errResultPaginatorCreation)

	ErrUnknownNode = errors.New(ErrUnknownNodePrefix)

	ErrDeploymentTerminalNoCancel    = errors.New(errDeploymentTerminalNoCancel)
	ErrDeploymentTerminalNoFail      = errors.New(errDeploymentTerminalNoFail)
	ErrDeploymentTerminalNoPause     = errors.New(errDeploymentTerminalNoPause)
	ErrDeploymentTerminalNoPromote   = errors.New(errDeploymentTerminalNoPromote)
	ErrDeploymentTerminalNoResume    = errors.New(errDeploymentTerminalNoResume)
	ErrDeploymentTerminalNoUnblock   = errors.New(errDeploymentTerminalNoUnblock)
	ErrDeploymentTerminalNoRun       = errors.New(errDeploymentTerminalNoRun)
	ErrDeploymentTerminalNoSetHealth = errors.New(errDeploymentTerminalNoSetHealth)
	ErrDeploymentRunningNoUnblock    = errors.New(errDeploymentRunningNoUnblock)

	ErrCSIClientRPCIgnorable  = errors.New("CSI client error (ignorable)")
	ErrCSIClientRPCRetryable  = errors.New("CSI client error (retryable)")
	ErrCSIVolumeMaxClaims     = errors.New("volume max claims reached")
	ErrCSIVolumeUnschedulable = errors.New("volume is currently unschedulable")
	ErrCSIPluginInUse         = errors.New("plugin in use")
)

// IsErrNoLeader returns whether the error is due to there being no leader.
func IsErrNoLeader(err error) bool {
	return err != nil && strings.Contains(err.Error(), errNoLeader)
}

// IsErrNoRegionPath returns whether the error is due to there being no path to
// the given region.
func IsErrNoRegionPath(err error) bool {
	return err != nil && strings.Contains(err.Error(), errNoRegionPath)
}

// IsErrTokenNotFound returns whether the error is due to the passed token not
// being resolvable.

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Confirm the wrapped plugin error is genuinely transient, then simply retry the operation
2. In code, detect it via strings.Contains(err.Error(), structs.ErrCSIClientRPCIgnorable.Error()) as the codebase does, since errors.Is does not survive RPC wrapping
3. Log at debug level and continue — the error is classified as ignorable by design
4. If it fires persistently, inspect the CSI plugin logs for the underlying cause

Example fix

// before
if err != nil {
    return fmt.Errorf("csi hook failed: %w", err)
}
// after
if err != nil {
    if strings.Contains(err.Error(), structs.ErrCSIClientRPCIgnorable.Error()) {
        return nil // benign, safe to continue
    }
    return fmt.Errorf("csi hook failed: %w", err)
}

// Check plugin health before RPC
plugins, _, _ := client.CSIPlugins().List(nil)
for _, p := range plugins.Plugins {
    if p.Provider == csiProvider && p.ControllersHealthy == 0 {
        return fmt.Errorf("no healthy controllers for %s", csiProvider)
    }
}

func isIgnorableCSIErr(err error) bool {
    return err != nil && strings.Contains(err.Error(), structs.ErrCSIClientRPCIgnorable.Error())
}

if err != nil {
    if isIgnorableCSIErr(err) {
        return nil // safe to continue
    }
    return fmt.Errorf("csi rpc failed: %w", err)
}

When it happens

Trigger: A CSI node/client plugin returns an error that Nomad classifies as ignorable during allocation hooks or client RPCs; callers like csi_hook.go match the message substring to decide retryability.

Common situations: A CSI plugin returns a transient error during an alloc runner hook; multiple CSI plugin instances respond inconsistently; plugin versions emit errors that are actually benign (e.g. already-detached volumes).

Related errors

• CSI.ControllerAttachVolume: VolumeID is required
• CSI.ControllerAttachVolume: ClientCSINodeID is required
• CSI.ControllerDetachVolume: VolumeID is required
• CSI.ControllerDetachVolume: ClientCSINodeID is required
• CSI.NodeDetachVolume: PluginID is required

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/e436a24f446d125a. Report an issue: GitHub.