hashicorp/nomad · error

failed deleting variable entry: %s

Error message

failed deleting variable entry: %s

What it means

Returned by svDeleteTxn when tx.Delete(TableVariables, sv) fails while removing the variable's VariableEncrypted object from the state store. Quota and lookup steps succeeded, but the actual row deletion failed, aborting the transaction with an ErrorResponse.

Source

Thrown at nomad/state/state_store_variables.go:434

	existingQuota, err := tx.First(TableVariablesQuotas, indexID, req.Var.Namespace)
	if err != nil {
		return req.ErrorResponse(idx, fmt.Errorf("variable quota lookup failed: %v", err))
	}

	// Track quota usage
	if existingQuota != nil {
		quotaUsed := existingQuota.(*structs.VariablesQuota)
		quotaUsed = quotaUsed.Copy()
		quotaUsed.Size -= min(quotaUsed.Size, int64(len(sv.Data)))
		quotaUsed.ModifyIndex = idx
		if err := tx.Insert(TableVariablesQuotas, quotaUsed); err != nil {
			return req.ErrorResponse(idx, fmt.Errorf("variable quota insert failed: %v", err))
		}
	}

	// Delete the variable and update the index table.
	if err := tx.Delete(TableVariables, sv); err != nil {
		return req.ErrorResponse(idx, fmt.Errorf("failed deleting variable entry: %s", err))
	}

	if err := tx.Insert(tableIndex, &IndexEntry{TableVariables, idx}); err != nil {
		return req.ErrorResponse(idx, fmt.Errorf("failed updating variable index: %s", err))
	}

	return req.SuccessResponse(idx, nil)
}

// WriteTxn is implemented by memdb.Txn to perform write operations.
type WriteTxn interface {
	ReadTxn
	Defer(func())
	Delete(table string, obj any) error
	DeleteAll(table, index string, args ...any) (int, error)
	DeletePrefix(table string, index string, prefix string) (bool, error)
	Insert(table string, obj any) error
}

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Read the wrapped %s error for the memdb cause
2. Retry the delete; aborting leaves the variable intact
3. Verify store integrity / restore from snapshot if recurring
4. Check for concurrent writers misusing the same txn

if err != nil && strings.Contains(err.Error(), "failed deleting variable entry") {
    // variable left intact; retry
    return retryWithBackoff(deleteFn)
}

When it happens

Trigger: VarDelete / svDeleteCASTxn on an existing variable where tx.Delete on TableVariables errors — memdb refusing the delete (e.g. object no longer matches, txn aborted, internal failure).

Common situations: Concurrent modification racing the delete within the same txn scope in custom tooling; corrupted table; aborted txns under load.

Related errors

• launch delete failed: %v
• variable quota insert failed: %v
• variable doesn't exist
• variable doesn't hold a lock
• index update failed: %v

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/986e06c1ee93883d. Report an issue: GitHub.