hashicorp/nomad · warning · ErrLockConflict

acquire conflict %w

Error message

acquire conflict %w

What it means

Lock.Acquire wraps ErrLockConflict when the variables/lock acquire HTTP call returns 409 Conflict, producing 'acquire conflict %w'. This is expected behavior when multiple instances contend for the same lock — the caller should interpret the wrapped ErrLockConflict via errors.Is rather than string matching.

Source

Thrown at api/locks.go:115

//	Acquire will make the actual call to acquire the lock over the variable using
//	the ttl in the Locks to create the VariableLock. It will return the
//	path of the variable holding the lock.
//
// Acquire returns the path to the variable holding the lock.
func (l *Locks) Acquire(ctx context.Context) (string, error) {

	var out Variable

	_, err := l.c.retryPut(ctx, "/v1/var/"+l.variable.Path+"?lock-acquire", l.variable, &out, &l.WriteOptions)
	if err != nil {
		callErr, ok := err.(UnexpectedResponseError)

		// http.StatusConflict means the lock is already held. This will happen
		// under the normal execution if multiple instances are fighting for the same lock and
		// doesn't disrupt the flow.
		if ok && callErr.statusCode == http.StatusConflict {
			return "", fmt.Errorf("acquire conflict %w", ErrLockConflict)
		}

		return "", err
	}

	l.variable.Lock = out.Lock

	return l.variable.Path, nil
}

// Release makes the call to release the lock over a variable, even if the ttl
// has not yet passed.
// In case of a call to release a non held lock, Release returns ErrLockConflict.
func (l *Locks) Release(ctx context.Context) error {
	var out Variable

	rv := &Variable{
		Lock: &VariableLock{

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Detect the conflict with errors.Is(err, api.ErrLockConflict) and implement retry-with-backoff or exit gracefully
2. Randomize/jitter acquisition attempts to reduce contention
3. Check the current lock holder (nomad var inspect) if the conflict seems stale
4. Ensure the lock is released on shutdown so holders don't linger

Example fix

// before
lockID, err := l.Acquire(ctx)
if err != nil { return err }
// after
lockID, err := l.Acquire(ctx)
if errors.Is(err, api.ErrLockConflict) {
    time.Sleep(jitter(2 * time.Second))
    return retryAcquire(ctx)
} else if err != nil {
    return err
}

func isLockConflict(err error) bool {
    return errors.Is(err, api.ErrLockConflict)
}

lockID, err := l.Acquire(ctx)
switch {
case errors.Is(err, api.ErrLockConflict):
    return retryWithBackoff(ctx)
case err != nil:
    return err
default:
    return useLock(ctx, lockID)
}

When it happens

Trigger: Two or more processes call api.Lock.Acquire for the same lock variable concurrently; the server already has a holder, so it replies HTTP 409.

Common situations: Highly available deployments racing at startup for leadership; lock TTL expired and re-acquired by another instance; retry loops without jitter hammering the same lock.

Related errors

• variable already holds a lock
• ErrLockConflict
• ErrVolumeNameExists
• ErrAllocBroadcasterClosed
• root key in use, cannot delete

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/c437fa2437bdf9f4. Report an issue: GitHub.