hashicorp/nomad · error

csi_plugins update error %s: %v

Error message

csi_plugins update error %s: %v

What it means

Wraps a memdb Insert failure when updateOrGCPlugin re-persists a still-in-use CSIPlugin with a new ModifyIndex. Unlike 2527 the plugin is not empty, so it is updated rather than deleted; the insert failed and fails the enclosing transaction.

Source

Thrown at nomad/state/state_store.go:1579

	if err := txn.Insert("index", &IndexEntry{TableCSIPlugins, index}); err != nil {
		return fmt.Errorf("index update failed: %v", err)
	}

	return nil
}

// updateOrGCPlugin updates a plugin but will delete it if the plugin is empty
func updateOrGCPlugin(index uint64, txn Txn, plug *structs.CSIPlugin) error {
	if plug.IsEmpty() {
		err := txn.Delete(TableCSIPlugins, plug)
		if err != nil {
			return fmt.Errorf("csi_plugins delete error: %v", err)
		}
	} else {
		plug.ModifyIndex = index
		err := txn.Insert(TableCSIPlugins, plug)
		if err != nil {
			return fmt.Errorf("csi_plugins update error %s: %v", plug.ID, err)
		}
	}
	return nil
}

// deleteJobFromPlugins removes the allocations of this job from any plugins the job is
// running, possibly deleting the plugin if it's no longer in use. It's called in DeleteJobTxn
func (s *StateStore) deleteJobFromPlugins(index uint64, txn Txn, job *structs.Job) error {
	ws := memdb.NewWatchSet()
	summary, err := s.JobSummaryByID(ws, job.Namespace, job.ID)
	if err != nil {
		return fmt.Errorf("error getting job summary: %v", err)
	}

	allocs, err := s.AllocsByJob(ws, job.Namespace, job.ID, false)
	if err != nil {
		return fmt.Errorf("error getting allocations: %v", err)
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Retry the triggering operation.
2. Inspect the wrapped plug.ID and %v error to identify the failing plugin and memdb cause.
3. Verify healthy Raft/state-store operation on the server leader; restart if degraded.
4. Upgrade Nomad if reproducible on your release.

if err != nil && strings.Contains(err.Error(), "csi_plugins update error") {
    pluginID := extractQuotedID(err) // parse %s from message if needed
    logger.Warn("csi plugin update failed", "plugin", pluginID, "err", err)
    return backoffRetry(triggeringOp)
}

When it happens

Trigger: Callers updatePluginForTerminalAlloc / updatePluginWithJobSummary / deleteJobFromPlugins / node upsert-delete paths reaching the else branch and txn.Insert(TableCSIPlugins, plug) erroring.

Common situations: Seen during alloc completion or job teardown for CSI workloads; generally an internal state-store failure rather than operator error.

Related errors

• csi_plugin lookup error: %s %v
• csi_plugins insert error: %v
• csi_plugins lookup failed: %v
• csi_plugins lookup error %s: %v
• volume insert: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/9cf824bd35525b89. Report an issue: GitHub.