What causes "conflicting operation over lock" in hashicorp/nomad?

Use errors.Is(err, api.ErrLockConflict) to detect it and retry acquire with backoff (honor DefaultLockDelay of 15s)

ErrLockConflict

ErrLockConflict

Error message

conflicting operation over lock

What it means

ErrLockConflict is the sentinel error returned when a lock operation (acquire, release, renew) cannot be performed because the caller is not the current holder of the lock. The Nomad server responds HTTP 409 Conflict and the client wraps this sentinel into the returned error.

Source

Thrown at api/locks.go:32

const (
	lockLeaseRenewalFactor = 0.7
	lockRetryBackoffFactor = 1.1

	// DefaultLockTTL is the default value used to maintain a lock before it needs to
	// be renewed. The actual value comes from the experience with Consul.
	DefaultLockTTL = 15 * time.Second

	// DefaultLockDelay is the default a lock will be blocked after the TTL
	// went by without any renews. It is intended to prevent split brain situations.
	// The actual value comes from the experience with Consul.
	DefaultLockDelay = 15 * time.Second
)

var (
	// ErrLockConflict is returned in case a lock operation can't be performed
	// because the caller is not the current holder of the lock.
	ErrLockConflict = errors.New("conflicting operation over lock")

	//LockNoPathErr is returned when no path is provided in the variable to be
	// used for the lease mechanism
	LockNoPathErr = errors.New("variable's path can't be empty")
)

// Locks returns a new handle on a lock for the given variable.
func (c *Client) Locks(wo WriteOptions, v Variable, opts ...LocksOption) (*Locks, error) {

	if v.Path == "" {
		return nil, LockNoPathErr
	}

	ttl, err := time.ParseDuration(v.Lock.TTL)
	if err != nil {
		return nil, err
	}

View on GitHub (pinned to 482b49bf1a)

Solutions

Use errors.Is(err, api.ErrLockConflict) to detect it and retry acquire with backoff (honor DefaultLockDelay of 15s)
Check whether your Lock session is still alive before releasing/renewing; re-create the session and re-acquire if expired
Ensure only one instance/leader attempts to hold a given lock path
Inspect the lock variable's Session value via the API to identify the current holder

Example fix

// before
_, err := lock.Acquire(q)
if err != nil { return err }
// after
_, err := lock.Acquire(q)
if errors.Is(err, api.ErrLockConflict) {
    time.Sleep(api.DefaultLockDelay)
    _, err = lock.Acquire(q)
}
if err != nil { return err }

Defensive patterns

Strategy: retry

Validate before calling

func retryableLockErr(err error) bool { return errors.Is(err, api.ErrLockConflict) }

Try / catch

_, err := lock.Acquire(nil)
if errors.Is(err, api.ErrLockConflict) {
    // wait and retry acquire with backoff
    time.Sleep(api.DefaultLockDelay)
    _, err = lock.Acquire(nil)
}

Prevention

Use errors.Is (not == or string match) to detect ErrLockConflict through %w wrapping
Retry acquire with backoff honoring DefaultLockDelay (15s)
Ensure only one leader/instance holds a given lock path
Monitor session health; recreate expired sessions before release/renew

When it happens

Trigger: Acquiring a lock already held by another session/instance; releasing or renewing a lock after your session expired or was invalidated; two processes contending for the same lock variable path.

Common situations: Split-brain or stale leader instances in a deployment tool; lock hold time exceeded and session reaped; multiple replicas starting simultaneously racing on the same lock path.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/306d9646e0883531. Report an issue: GitHub.