elsa-workflows/elsa-core · warning · InvalidOperationException

Drain already in progress; second invocation rejected

Error message

Drain already in progress; second invocation rejected (trigger={trigger}).

What it means

DrainOrchestrator.DrainAsync serializes drain operations with a lock and a _drainInProgress flag. When a drain is already running, a second invocation is rejected with InvalidOperationException, except when the caller passes DrainTrigger.OperatorForce and a previous outcome exists, in which case the cached outcome is returned. This prevents concurrent drain passes from racing.

Solutions

Wait for the in-flight drain to complete before invoking again; check the orchestrator's state/outcome first.
If intentional, pass DrainTrigger.OperatorForce to obtain the cached previous outcome instead of throwing.
Ensure DrainAsync is invoked from a single place (singleton coordinator) rather than concurrently from multiple schedulers.
Wrap the call in try/catch for InvalidOperationException and treat it as 'drain already running' rather than a fault.

Example fix

// before
await drainOrchestrator.DrainAsync(DrainTrigger.Shutdown, ct);

// after
try
{
    await drainOrchestrator.DrainAsync(DrainTrigger.Shutdown, ct);
}
catch (InvalidOperationException) when (ex.Message.Contains("Drain already in progress"))
{
    logger.LogInformation("Drain already in progress; skipping this invocation.");
}

Defensive patterns

Strategy: try-catch

Validate before calling

// No pre-call API to check in-flight state; rely on single-caller design.

Try / catch

try
{
    await drainOrchestrator.DrainAsync(trigger, ct);
}
catch (InvalidOperationException ex) when (ex.Message.StartsWith("Drain already in progress"))
{
    logger.LogInformation("Drain skipped: already running.");
}

Prevention

Invoke DrainAsync from a single coordinator
Never overlap scheduled and manual drains
Use OperatorForce when a cached outcome is acceptable

When it happens

Trigger: Invoking DrainAsync (public) while another DrainAsync call is still executing (e.g. two operators or a scheduler overlapping an in-flight drain) without DrainTrigger.OperatorForce.

Common situations: A scheduled drain overlapping a manual operator-triggered drain; retry logic re-invoking DrainAsync while the first attempt is still running; multiple instances of an admin job sharing the same orchestrator singleton.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of elsa-workflows/elsa-core@fe9217bdfa (2026-09-13). Data as JSON: /api/errors/5046341d2061491e. Report an issue: GitHub.

Appendix: source

Thrown at src/modules/Elsa.Workflows.Runtime/Services/DrainOrchestrator.cs:86

        _cycles = cycles;
        _scopeFactory = scopeFactory;
        _options = options;
        _hostOptions = hostOptions;
        _clock = clock;
        _identityGenerator = identityGenerator;
        _logger = logger;
    }

    /// <inheritdoc />
    public async ValueTask<DrainOutcome> DrainAsync(DrainTrigger trigger, CancellationToken cancellationToken = default)
    {
        lock (_sync)
        {
            if (_drainInProgress)
            {
                if (trigger == DrainTrigger.OperatorForce && _previousOutcome is not null)
                    return _previousOutcome with { WasCached = true };
                throw new InvalidOperationException($"Drain already in progress; second invocation rejected (trigger={trigger}).");
            }
            if (_previousOutcome is not null)
            {
                if (trigger == DrainTrigger.OperatorForce) return _previousOutcome with { WasCached = true };
                throw new InvalidOperationException("Drain already completed in this generation; subsequent non-force invocations are rejected.");
            }
            _drainInProgress = true;
        }

        var startedAt = _clock.UtcNow;
        var deadline = ComputeEffectiveDeadline(trigger);
        var sw = Stopwatch.StartNew();
        TimeSpan pausePhase = TimeSpan.Zero;
        TimeSpan waitPhase = TimeSpan.Zero;

        try
        {
            await _signal.BeginDrainAsync(cancellationToken);

View on GitHub (pinned to fe9217bdfa)