argoproj/argo-workflows · error

couldn't decode locks for session: %w

Error message

couldn't decode locks for session: %w

What it means

TryAcquire calls needDBSession to detect database-backed locks among the lock keys; if a key cannot be decoded the call aborts with 'couldn't decode locks for session'. A lock key string handed to the manager is not a valid encoded lock name.

Source

Thrown at workflow/sync/sync_manager.go:481

	holderKey := getHolderKey(wf, nodeName)

	lockKeys := make([]string, len(syncItems))
	for i, syncItem := range syncItems {
		syncLockName, lockNameErr := syncItem.lockName(wf.Namespace)
		if lockNameErr != nil {
			return false, false, "", failedLockName, fmt.Errorf("requested configuration is invalid: %w", lockNameErr)
		}
		sm.log.WithField("syncLockName", syncLockName).Info(ctx, "TryAcquire")
		lockKeys[i] = syncLockName.String(ctx)
	}

	if ok, msg, prepLockName, prepErr := sm.prepAcquire(ctx, wf, holderKey, syncItems, lockKeys); !ok {
		return false, false, msg, prepLockName, prepErr
	}

	needDB, err := needDBSession(ctx, lockKeys)
	if err != nil {
		return false, false, "", failedLockName, fmt.Errorf("couldn't decode locks for session: %w", err)
	}
	if needDB && sm.dbInfo.SessionProxy == nil {
		return false, false, "", failedLockName, fmt.Errorf("synchronization database session is not available")
	}
	if needDB {
		var updated bool
		var already bool
		var msg string
		var newly []*acquiredLock
		backoff := dbRetryBackoff
		// tryAcquireImpl mutates wf.Status.Synchronization in memory before the
		// transaction commits. Snapshot it and roll each failed attempt back, so
		// that attempts are independent and an error return leaves the caller's
		// status exactly as it was - otherwise an abort leaves a Holding entry
		// for a row the database rolled back, which would be persisted and then
		// failed as a stale hold on the next controller restart.
		syncStatus := wf.Status.Synchronization.DeepCopy()
		attempt := 0

View on GitHub (pinned to 35bff19146)

Solutions

1. Log and inspect the offending lock key to see why decoding fails
2. Align controller versions so lock-name encoding matches
3. Fix or regenerate the sync configuration producing the malformed name
4. Clear the stale lock record so the key is rebuilt from a valid reference

if _, err := sync.DecodeLockName(ctx, lockKey); err != nil {
    return fmt.Errorf("skip undecodable lock key %q: %w", lockKey, err)
}

func decodableLockKey(ctx context.Context, key string) bool {
    _, err := sync.DecodeLockName(ctx, key)
    return err == nil
}

if err != nil && strings.Contains(err.Error(), "couldn't decode locks for session") { /* quarantine the key, rebuild from spec */ }

When it happens

Trigger: A lockKey produced from syncItem.lockName cannot be parsed during session detection — corrupted or unrecognized lock-name format reaching needDBSession.

Common situations: Mixed-version clusters where lock names were written in a format the current controller cannot decode; corrupted sync ConfigMaps; custom sync implementations emitting non-standard keys.

Related errors

• invalid lock kind %s when initializing semaphore
• cannot re-establish %s %q held by workflow %s/%s at startup:
• cannot acquire lock from nil Synchronization
• requested configuration is invalid: %w
• synchronization database session is not available

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/56fe9efe79877315. Report an issue: GitHub.