App-vNext/Polly · error · BulkheadRejectedException

The bulkhead semaphore and queue are full and execution was

Error message

The bulkhead semaphore and queue are full and execution was rejected.

What it means

Thrown as a BulkheadRejectedException when the bulkhead policy cannot admit a new execution because both the parallelization slots and the queue are saturated. The engine calls maxQueuedActionsSemaphore.Wait(TimeSpan.Zero, ...) — a non-blocking probe — and if no queue slot is immediately available, the onBulkheadRejected callback fires and the exception is thrown. This is an intentional flow-control mechanism, not a bug.

Source

Thrown at src/Polly/Bulkhead/BulkheadEngine.cs:18

#nullable enable

namespace Polly.Bulkhead;

internal static class BulkheadEngine
{
    internal static TResult Implementation<TResult>(
        Func<Context, CancellationToken, TResult> action,
        Context context,
        Action<Context> onBulkheadRejected,
        SemaphoreSlim maxParallelizationSemaphore,
        SemaphoreSlim maxQueuedActionsSemaphore,
        CancellationToken cancellationToken)
    {
        if (!maxQueuedActionsSemaphore.Wait(TimeSpan.Zero, cancellationToken))
        {
            onBulkheadRejected(context);
            throw new BulkheadRejectedException();
        }

        try
        {
            maxParallelizationSemaphore.Wait(cancellationToken);
            try
            {
                return action(context, cancellationToken);
            }
            finally
            {
                SafeRelease(maxParallelizationSemaphore);
            }
        }
        finally
        {
            SafeRelease(maxQueuedActionsSemaphore);
        }

View on GitHub (pinned to d0e46bdb1e)

Solutions

Increase maxQueuingActions (and/or maxParallelization) in the Policy.Bulkhead(...) call to absorb larger bursts
Wrap the Execute call in a try/catch for BulkheadRejectedException and return a fallback or HTTP 503
Chain a retry-with-backoff policy in front of the bulkhead so rejected requests are re-attempted after a delay
Apply client-side rate limiting upstream so the queue never fills to capacity

Example fix

// before
var policy = Policy.Bulkhead(10, 5);
var result = policy.Execute(() => CallService()); // throws when queue full

// after
var policy = Policy.Bulkhead(10, 20);
try
{
    var result = policy.Execute(() => CallService());
}
catch (BulkheadRejectedException)
{
    return StatusCode(StatusCodes.Status503ServiceUnavailable, "At capacity");
}

Defensive patterns

Strategy: try-catch

Validate before calling

// Check bulkhead capacity before executing
if (policy.BulkheadAvailableCount == 0 && policy.QueueAvailableCount == 0)
{
    // Bulkhead will reject — handle proactively
    return fallbackResult;
}
var result = policy.Execute(() => CallService());

Type guard

public static bool CanAdmitRequest(BulkheadPolicy policy)
{
    return policy.BulkheadAvailableCount > 0 || policy.QueueAvailableCount > 0;
}

Try / catch

try
{
    var result = policy.Execute(() => CallService());
}
catch (BulkheadRejectedException ex)
{
    logger.LogWarning(ex, "Bulkhead rejected execution");
    return StatusCode(StatusCodes.Status503ServiceUnavailable, "Service at capacity");
}

Prevention

Size maxParallelization and maxQueuingActions based on load testing under realistic traffic
Monitor BulkheadAvailableCount and QueueAvailableCount to detect saturation before rejection
Chain a retry-with-jitter policy so transient rejections self-heal
Consider a fallback policy to return a default value instead of propagating the exception

When it happens

Trigger: Calling policy.Execute(...) or policy.ExecuteAsync(...) when in-flight concurrent executions already equal maxParallelization AND waiting requests already equal maxQueuingActions. The zero-timeout semaphore Wait returns false at BulkheadEngine.cs:18, triggering immediate rejection.

Common situations: Downstream service latency spike consuming all parallelization slots while requests pile up in the queue. Traffic burst exceeding maxParallelization + maxQueuingActions combined. Under-provisioned bulkhead limits chosen during development that do not match production load.

Related errors

AI-assisted analysis of App-vNext/Polly@d0e46bdb1e (2026-08-13). Data as JSON: /api/errors/d00984f4437be3e9. Report an issue: GitHub.