hashicorp/nomad · critical

Task group volume claim insert failed: %v

Error message

Task group volume claim insert failed: %v

What it means

This error wraps a memdb transaction failure that occurred while inserting a TaskGroupHostVolumeClaim row into the state store. txn.Insert fails only for low-level store problems (invalid object, txn corruption, schema mismatch), since claim-vs-existing conflicts are handled as updates earlier in the function. It surfaces during UpsertTaskGroupHostVolumeClaim or when sticky volume claims are reconciled from an allocation update.

Source

Thrown at nomad/state/state_store_task_group_volume_claims.go:58

		existing = existingRaw.(*structs.TaskGroupHostVolumeClaim)
	}

	if existing != nil {
		// do allocation ID and volume ID match?
		if existing.ClaimedByAlloc(claim) {
			return nil
		}

		claim.CreateIndex = existing.CreateIndex
		claim.ModifyIndex = index
	} else {
		claim.CreateIndex = index
		claim.ModifyIndex = index
	}

	// Insert the claim into the table.
	if err := txn.Insert(TableTaskGroupHostVolumeClaim, claim); err != nil {
		return fmt.Errorf("Task group volume claim insert failed: %v", err)
	}

	// Perform the index table update to mark the new insert.
	if err := txn.Insert(tableIndex, &IndexEntry{TableTaskGroupHostVolumeClaim, index}); err != nil {
		return fmt.Errorf("index update failed: %v", err)
	}

	return nil
}

// GetTaskGroupHostVolumeClaim returns a volume claim that matches the namespace,
// job id and task group name (there can be only one)
func (s *StateStore) GetTaskGroupHostVolumeClaim(ws memdb.WatchSet, namespace, jobID, taskGroupName, volumeID string) (*structs.TaskGroupHostVolumeClaim, error) {
	txn := s.db.ReadTxn()

	watchCh, existing, err := txn.FirstWatch(TableTaskGroupHostVolumeClaim, indexID, namespace, jobID, taskGroupName, volumeID)
	if err != nil {
		return nil, fmt.Errorf("Task group volume claim lookup failed: %v", err)

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Retry the Raft write; transient txn issues usually clear on retry
2. Check server logs for preceding memdb/schema errors to identify the underlying %v cause
3. Restart the Nomad server if the state store is believed corrupt (state is reloaded from Raft snapshots)
4. Upgrade Nomad if running a version with known volume-claim bugs

if err := store.UpsertTaskGroupHostVolumeClaim(idx, claims); err != nil {
    if strings.Contains(err.Error(), "insert failed") {
        // low-level store error: backoff then retry once
        return retryAfterBackoff()
    }
    return err
}

When it happens

Trigger: txn.Insert(TableTaskGroupHostVolumeClaim, claim) returns an error during UpsertTaskGroupHostVolumeClaim or updateStickyVolumeClaimsFromAlloc; practically only when the in-memory store is corrupt or the object violates the table schema.

Common situations: Corrupted Raft/state store requiring restart; a Nomad bug or version-mismatch writing an incompatible claim struct; disk/memory pressure corrupting memdb.

Related errors

• index update failed: %v
• csi_plugin lookup error: %s %v
• csi_plugins insert error: %v
• csi_plugins lookup failed: %v
• csi_plugins lookup error %s: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/9861e3560c00a50c. Report an issue: GitHub.