hashicorp/nomad · error

csi_plugin lookup failed: %s %v

Error message

csi_plugin lookup failed: %s %v

What it means

CSIPluginByIDTxn fetches a single CSIPlugin by ID using txn.FirstWatch on the csi_plugins table's "id" index. This error wraps a FirstWatch failure — a memdb-level error (invalid index/table), NOT a missing plugin (that returns nil obj, nil err). Callers like csiVolumeDenormalizePluginsTxn and CSIPluginByID re-wrap it.

Source

Thrown at nomad/state/state_store.go:3202

}

// CSIPluginByID returns a named CSIPlugin. This method creates a new
// transaction so you should not call it from within another transaction.
func (s *StateStore) CSIPluginByID(ws memdb.WatchSet, id string) (*structs.CSIPlugin, error) {
	txn := s.db.ReadTxn()
	plugin, err := s.CSIPluginByIDTxn(txn, ws, id)
	if err != nil {
		return nil, err
	}
	return plugin, nil
}

// CSIPluginByIDTxn returns a named CSIPlugin
func (s *StateStore) CSIPluginByIDTxn(txn Txn, ws memdb.WatchSet, id string) (*structs.CSIPlugin, error) {

	watchCh, obj, err := txn.FirstWatch(TableCSIPlugins, "id", id)
	if err != nil {
		return nil, fmt.Errorf("csi_plugin lookup failed: %s %v", id, err)
	}

	ws.Add(watchCh)

	if obj != nil {
		return obj.(*structs.CSIPlugin), nil
	}
	return nil, nil
}

// CSIPluginDenormalize returns a CSIPlugin with allocation details. Always called on a copy of the plugin.
func (s *StateStore) CSIPluginDenormalize(ws memdb.WatchSet, plug *structs.CSIPlugin) (*structs.CSIPlugin, error) {
	txn := s.db.ReadTxn()
	return s.CSIPluginDenormalizeTxn(txn, ws, plug)
}

func (s *StateStore) CSIPluginDenormalizeTxn(txn Txn, ws memdb.WatchSet, plug *structs.CSIPlugin) (*structs.CSIPlugin, error) {
	if plug == nil {

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Read the wrapped %v error for the underlying memdb cause.
2. Restart the server agent to rebuild state from Raft snapshots.
3. Verify cluster-wide Nomad version consistency.
4. Escalate to Raft recovery if the error persists across restarts.

func isMissingPlugin(err error) bool {
    // nil plugin (not an error) means absent; error containing this text means memdb failure
    return err != nil && strings.Contains(err.Error(), "csi_plugin lookup failed")
}

plug, err := store.CSIPluginByID(ws, ns, id)
switch {
case err == nil && plug != nil: // found
case err == nil && plug == nil: // absent — treat as not-found, not error
case strings.Contains(err.Error(), "lookup failed"): // memdb failure — escalate
    return err
}

When it happens

Trigger: Any CSIPluginByID/CSIPluginByIDTxn call where txn.FirstWatch(TableCSIPlugins, "id", id) returns an error — internal memdb failure rather than absent data.

Common situations: Corrupt state store, incompatible state DB from a different Nomad version, or memory pressure damaging the memdb table.

Related errors

• error parsing: root should be an object
• plugin lookup error: %s %v
• cannot specify Accessor ID
• failed to read dynamic plugin registry state: %v
• error getting plugin: %s, %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/8f59aad9ebfa2871. Report an issue: GitHub.