microsoft/garnet · error · GarnetException

Failed to acquire inProgress lock at

Error message

Failed to acquire inProgress lock at {nameof(PostBatchKeyConsistentReadCallback)}

What it means

Thrown by ReplicaReadSessionContext.PostBatchKeyConsistentReadCallback when TryReadLock() on the 'inProgress' lock fails. This is the post-read validation step for a batch: it validates each key's hash against the consistency manager and propagates the batch context back to the session. The read lock is unavailable because Dispose holds the write lock.

Solutions

1. Ensure dispose does not run while batch reads are pending (drain first).
2. Guard the callback with a disposed check and catch GarnetException to treat the batch as invalidated.
3. Review the session close path to guarantee all PostBatch callbacks have completed.
4. Log the lock failure to correlate with dispose timing during incident analysis.

Example fix

// before
bool ok = session.PostBatchKeyConsistentReadCallback(keyCount);

// after
bool ok;
try { ok = session.PostBatchKeyConsistentReadCallback(keyCount); }
catch (GarnetException) { ok = false; /* session disposed; batch invalid */ }

if (session.IsDisposed) return false;
return session.PostBatchKeyConsistentReadCallback(keyCount);

try { return session.PostBatchKeyConsistentReadCallback(keyCount); }
catch (GarnetException ex) when (ex.Message.Contains("inProgress lock"))
{ return false; /* batch invalidated by session disposal */ }

When it happens

Trigger: The batch post-validation callback runs after a batch read completes, but the session is being disposed concurrently. The non-blocking TryReadLock fails and the method throws rather than blocking.

Common situations: Session teardown racing with batch-read completion; a cancellation path that disposes without waiting for in-flight batch reads; a re-entrancy bug.

Related errors

• Failed to acquire inProgress lock at
• Failed to acquire inProgress lock at
• Failed to acquire inProgress lock at
• Replay header type not supported!
• Type not supported

AI-assisted analysis of microsoft/garnet@951b0fc683 (2026-08-13). Data as JSON: /api/errors/6fca4fdfdc6e4a0d. Report an issue: GitHub.

Appendix: source

Thrown at libs/server/AOF/ReadConsistency/ReplicaReadSessionContext.cs:239

                    consistencyManager.PreBatchKeyConsistentRead(key.ReadOnlySpan, ref batchReadContext, readTimeout, consistentReadCts.Token, out var hash);
                    keyHashCache[i] = hash;
                }
            }
            finally
            {
                inProgress.ReadUnlock();
            }
        }

        /// <summary>
        /// Validate keys have not changed after reading a key batch.
        /// </summary>
        /// <param name="keyCount"></param>
        /// <returns></returns>
        public bool PostBatchKeyConsistentReadCallback(int keyCount)
        {
            if (!inProgress.TryReadLock())
                throw new GarnetException($"Failed to acquire inProgress lock at {nameof(PostBatchKeyConsistentReadCallback)}");
            try
            {
                var consistencyManager = appendOnlyFile.readConsistencyManager;
                for (var i = 0; i < keyCount; i++)
                {
                    var hash = keyHashCache[i];
                    if (!consistencyManager.PostBatchKeyConsistentReadValidate(hash, ref batchReadContext))
                        return false;
                }

                // Propagate batch context back to session context to maintain prefix consistency
                // for subsequent single-key reads across different sublogs.
                replicaReadContext.maximumSessionSequenceNumber = batchReadContext.maximumSessionSequenceNumber;
                replicaReadContext.lastVirtualSublogIdx = batchReadContext.lastVirtualSublogIdx;
                replicaReadContext.lastHash = batchReadContext.lastHash;

                return true;
            }

View on GitHub (pinned to 951b0fc683)