dotnet/orleans · error · InvalidOperationException

Recovered {name} count does not match. Written: {written.Cou

Error message

Recovered {name} count does not match. Written: {written.Count}, recovered: {recovered.Count}.

What it means

An InvalidOperationException thrown by the list-overload of EnsureEqual during recovery validation when a recovered durable collection (inventory, events, work queue, tags) has a different element count than what was written. It indicates the journal was not replayed completely or an entry was lost/added during the write-recover cycle. The message names the collection and both counts.

Source

Thrown at samples/JournalingAzureBlobJson/JournalingAzureBlobJson/Program.cs:124

        {
            throw new InvalidOperationException("The grain did not reactivate after DeactivateOnIdle.");
        }

        EnsureEqual("inventory", written.Inventory, recovered.Inventory, InventoryEntryEquals);
        EnsureEqual("events", written.Events, recovered.Events, JournalEventEquals);
        EnsureEqual("work queue", written.WorkQueue, recovered.WorkQueue, WorkItemEquals);
        EnsureEqual("tags", written.Tags, recovered.Tags, static (left, right) => string.Equals(left, right, StringComparison.Ordinal));
        EnsureEqual("balance", written.Balance, recovered.Balance, AccountBalanceEquals);
        EnsureEqual("profile", written.Profile, recovered.Profile, ProfileStateEquals);
        EnsureEqual("completion status", written.CompletionStatus, recovered.CompletionStatus, static (left, right) => left == right);
        EnsureEqual("receipt", written.Receipt, recovered.Receipt, static (left, right) => left == right);
    }

    private static void EnsureEqual<T>(string name, IReadOnlyList<T> written, IReadOnlyList<T> recovered, Func<T, T, bool> equals)
    {
        if (written.Count != recovered.Count)
        {
            throw new InvalidOperationException($"Recovered {name} count does not match. Written: {written.Count}, recovered: {recovered.Count}.");
        }

        for (var i = 0; i < written.Count; i++)
        {
            if (!equals(written[i], recovered[i]))
            {
                throw new InvalidOperationException($"Recovered {name} item {i} does not match. Written: {Serialize(written[i])}, recovered: {Serialize(recovered[i])}.");
            }
        }
    }

    private static void EnsureEqual<T>(string name, T written, T recovered, Func<T, T, bool> equals)
    {
        if (!equals(written, recovered))
        {
            throw new InvalidOperationException($"Recovered {name} does not match. Written: {Serialize(written)}, recovered: {Serialize(recovered)}.");
        }
    }

View on GitHub (pinned to fca799fa70)

Solutions

1. Ensure WriteStateAsync() has completed before deactivating the grain (the sample calls it at the end of RunScenario).
2. Inspect the raw WAL blob (printed by the sample) to confirm all journal entries are present and valid JSONL.
3. Check the durable-collection codec/source generation — a missing [GenerateSerializer]/[JsonSerializable] can silently drop items.

Example fix

// before (no guard that state flushed)
return CreateSummary();

// after (explicit flush before summarizing)
await WriteStateAsync();
return CreateSummary();

static void RequireCount<T>(string name, IReadOnlyList<T> written, IReadOnlyList<T> recovered) {
    if (written.Count != recovered.Count)
        throw new InvalidOperationException($"{name} count mismatch: {written.Count} vs {recovered.Count}");
}

When it happens

Trigger: After deactivation+recovery, written.<collection>.Count != recovered.<collection>.Count for one of the validated lists (EnsureEqual is called on Inventory, Events, WorkQueue, Tags). The first divergence detected throws.

Common situations: A journaling bug (entries not flushed before deactivation, checkpoint/WAL truncation), a serialization mismatch dropping items, or premature deactivation before WriteStateAsync completed. A schema change between write and recovery.

Related errors

• Recovered {name} item {i} does not match. Written: {Serializ
• The grain did not reactivate after DeactivateOnIdle.
• Recovered {name} does not match. Written: {Serialize(written
• The frontend cant support more than 6 silos
• Withdrawing {amount} credits from account "{this.GetPrimaryK

AI-assisted analysis of dotnet/orleans@fca799fa70 (2026-08-13). Data as JSON: /api/errors/e980655d01132ecd. Report an issue: GitHub.