dotnet/orleans · error · InvalidOperationException

The grain did not reactivate after DeactivateOnIdle.

Error message

The grain did not reactivate after DeactivateOnIdle.

What it means

An InvalidOperationException thrown by ValidateRecovery in the JournalingAzureBlobJson sample after the journaled grain is deactivated and re-read. The check compares the ActivationId written before deactivation with the one recovered after; if they are equal, the grain was not actually reactivated (DeactivateOnIdle did not produce a new activation), so the recovery test is invalid. This is a self-check that the durability round-trip exercised a fresh activation.

Source

Thrown at samples/JournalingAzureBlobJson/JournalingAzureBlobJson/Program.cs:107

        Console.WriteLine();
        Console.WriteLine($"Raw JSONL blob contents from {settings.ContainerName}/{walBlobName}:");
        Console.WriteLine(await DownloadBlobAsText(blob));

        await host.StopAsync();
        return 0;
    }

    private static async Task<string> DownloadBlobAsText(AppendBlobClient blob)
    {
        var result = await blob.DownloadContentAsync();
        return Encoding.UTF8.GetString(result.Value.Content.ToArray());
    }

    private static void ValidateRecovery(JournaledSampleSummary written, JournaledSampleSummary recovered)
    {
        if (written.ActivationId == recovered.ActivationId)
        {
            throw new InvalidOperationException("The grain did not reactivate after DeactivateOnIdle.");
        }

        EnsureEqual("inventory", written.Inventory, recovered.Inventory, InventoryEntryEquals);
        EnsureEqual("events", written.Events, recovered.Events, JournalEventEquals);
        EnsureEqual("work queue", written.WorkQueue, recovered.WorkQueue, WorkItemEquals);
        EnsureEqual("tags", written.Tags, recovered.Tags, static (left, right) => string.Equals(left, right, StringComparison.Ordinal));
        EnsureEqual("balance", written.Balance, recovered.Balance, AccountBalanceEquals);
        EnsureEqual("profile", written.Profile, recovered.Profile, ProfileStateEquals);
        EnsureEqual("completion status", written.CompletionStatus, recovered.CompletionStatus, static (left, right) => left == right);
        EnsureEqual("receipt", written.Receipt, recovered.Receipt, static (left, right) => left == right);
    }

    private static void EnsureEqual<T>(string name, IReadOnlyList<T> written, IReadOnlyList<T> recovered, Func<T, T, bool> equals)
    {
        if (written.Count != recovered.Count)
        {
            throw new InvalidOperationException($"Recovered {name} count does not match. Written: {written.Count}, recovered: {recovered.Count}.");
        }

View on GitHub (pinned to fca799fa70)

Solutions

Increase the post-deactivation delay or poll until a new ActivationId is observed before validating recovery.
Ensure DeactivateOnIdle actually completes — call a method that forces reactivation or wait for the deactivation callback.
Confirm no other call is keeping the activation alive (which would cancel idle deactivation).

Example fix

// before
await grain.Deactivate();
await Task.Delay(TimeSpan.FromMilliseconds(500));
var recovered = await grain.GetSummary();

// after (poll for a fresh activation)
await grain.Deactivate();
var writtenId = written.ActivationId;
Guid recoveredId;
JournaledSampleSummary recovered;
do {
    await Task.Delay(TimeSpan.FromMilliseconds(250));
    recovered = await grain.GetSummary();
    recoveredId = recovered.ActivationId;
} while (recoveredId == writtenId);

Defensive patterns

Strategy: validation

Validate before calling

var writtenId = written.ActivationId;
JournaledSampleSummary recovered;
do {
    await Task.Delay(TimeSpan.FromMilliseconds(250));
    recovered = await grain.GetSummary();
} while (recovered.ActivationId == writtenId);

Prevention

Wait long enough (or poll) for the activation to be torn down and recreated after DeactivateOnIdle.
Avoid keeping the activation alive with concurrent calls during deactivation.
Treat this exception as a test-harness timing bug, not a production failure.

When it happens

Trigger: grain.Deactivate() (which calls DeactivateOnIdle) is followed by only a short delay (500ms), then GetSummary() returns the same ActivationId — meaning the old activation was reused or not yet torn down before the summary was captured.

Common situations: The 500ms delay is too short on a slow machine so the activation hasn't been collected yet. Deactivation timing in Orleans is best-effort; under load the activation may persist longer. A grain-activation cache/call-interceptor returning the cached summary.

Related errors

AI-assisted analysis of dotnet/orleans@fca799fa70 (2026-08-13). Data as JSON: /api/errors/84e5d736a4044df9. Report an issue: GitHub.