hashicorp/nomad · critical

Task group volume claim insert failed

Error message

Task group volume claim insert failed: %v

What it means

This error wraps a memdb transaction failure that occurred while inserting a TaskGroupHostVolumeClaim row into the state store. txn.Insert fails only for low-level store problems (invalid object, txn corruption, schema mismatch), since claim-vs-existing conflicts are handled as updates earlier in the function. It surfaces during UpsertTaskGroupHostVolumeClaim or when sticky volume claims are reconciled from an allocation update.

Solutions

1. Retry the Raft write; transient txn issues usually clear on retry
2. Check server logs for preceding memdb/schema errors to identify the underlying %v cause
3. Restart the Nomad server if the state store is believed corrupt (state is reloaded from Raft snapshots)
4. Upgrade Nomad if running a version with known volume-claim bugs

if err := store.UpsertTaskGroupHostVolumeClaim(idx, claims); err != nil {
    if strings.Contains(err.Error(), "insert failed") {
        // low-level store error: backoff then retry once
        return retryAfterBackoff()
    }
    return err
}

When it happens

Trigger: txn.Insert(TableTaskGroupHostVolumeClaim, claim) returns an error during UpsertTaskGroupHostVolumeClaim or updateStickyVolumeClaimsFromAlloc; practically only when the in-memory store is corrupt or the object violates the table schema.

Common situations: Corrupted Raft/state store requiring restart; a Nomad bug or version-mismatch writing an incompatible claim struct; disk/memory pressure corrupting memdb.

Related errors

• ACL binding rule insert failed
• ACL binding rule lookup failed
• ACL policy lookup failed
• ACL role deletion failed
• ACL role insert failed

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/9861e3560c00a50c. Report an issue: GitHub.

Appendix: source

Thrown at nomad/state/state_store_task_group_volume_claims.go:58

		existing = existingRaw.(*structs.TaskGroupHostVolumeClaim)
	}

	if existing != nil {
		// do allocation ID and volume ID match?
		if existing.ClaimedByAlloc(claim) {
			return nil
		}

		claim.CreateIndex = existing.CreateIndex
		claim.ModifyIndex = index
	} else {
		claim.CreateIndex = index
		claim.ModifyIndex = index
	}

	// Insert the claim into the table.
	if err := txn.Insert(TableTaskGroupHostVolumeClaim, claim); err != nil {
		return fmt.Errorf("Task group volume claim insert failed: %v", err)
	}

	// Perform the index table update to mark the new insert.
	if err := txn.Insert(tableIndex, &IndexEntry{TableTaskGroupHostVolumeClaim, index}); err != nil {
		return fmt.Errorf("index update failed: %v", err)
	}

	return nil
}

// GetTaskGroupHostVolumeClaim returns a volume claim that matches the namespace,
// job id and task group name (there can be only one)
func (s *StateStore) GetTaskGroupHostVolumeClaim(ws memdb.WatchSet, namespace, jobID, taskGroupName, volumeID string) (*structs.TaskGroupHostVolumeClaim, error) {
	txn := s.db.ReadTxn()

	watchCh, existing, err := txn.FirstWatch(TableTaskGroupHostVolumeClaim, indexID, namespace, jobID, taskGroupName, volumeID)
	if err != nil {
		return nil, fmt.Errorf("Task group volume claim lookup failed: %v", err)

View on GitHub (pinned to 482b49bf1a)