dotnet/orleans · error · InvalidOperationException

Recovered {name} item {i} does not match. Written: {Serializ

Error message

Recovered {name} item {i} does not match. Written: {Serialize(written[i])}, recovered: {Serialize(recovered[i])}.

What it means

An InvalidOperationException thrown by EnsureEqual's list loop during recovery validation when the recovered collection has the correct count but an individual element differs from what was written. The message includes the collection name, the item index, and the serialized forms of both the written and recovered values for precise comparison.

Source

Thrown at samples/JournalingAzureBlobJson/JournalingAzureBlobJson/Program.cs:131

        EnsureEqual("tags", written.Tags, recovered.Tags, static (left, right) => string.Equals(left, right, StringComparison.Ordinal));
        EnsureEqual("balance", written.Balance, recovered.Balance, AccountBalanceEquals);
        EnsureEqual("profile", written.Profile, recovered.Profile, ProfileStateEquals);
        EnsureEqual("completion status", written.CompletionStatus, recovered.CompletionStatus, static (left, right) => left == right);
        EnsureEqual("receipt", written.Receipt, recovered.Receipt, static (left, right) => left == right);
    }

    private static void EnsureEqual<T>(string name, IReadOnlyList<T> written, IReadOnlyList<T> recovered, Func<T, T, bool> equals)
    {
        if (written.Count != recovered.Count)
        {
            throw new InvalidOperationException($"Recovered {name} count does not match. Written: {written.Count}, recovered: {recovered.Count}.");
        }

        for (var i = 0; i < written.Count; i++)
        {
            if (!equals(written[i], recovered[i]))
            {
                throw new InvalidOperationException($"Recovered {name} item {i} does not match. Written: {Serialize(written[i])}, recovered: {Serialize(recovered[i])}.");
            }
        }
    }

    private static void EnsureEqual<T>(string name, T written, T recovered, Func<T, T, bool> equals)
    {
        if (!equals(written, recovered))
        {
            throw new InvalidOperationException($"Recovered {name} does not match. Written: {Serialize(written)}, recovered: {Serialize(recovered)}.");
        }
    }

    private static bool InventoryEntryEquals(InventoryEntry left, InventoryEntry right)
        => string.Equals(left.Key, right.Key, StringComparison.Ordinal)
            && InventoryItemEquals(left.Value, right.Value);

    private static bool InventoryItemEquals(InventoryItem left, InventoryItem right)
        => string.Equals(left.Sku, right.Sku, StringComparison.Ordinal)

View on GitHub (pinned to fca799fa70)

Solutions

1. Compare the serialized written vs recovered values in the error message to identify the differing field.
2. Verify every field has a serialization attribute ([Id]/source-gen) and the type is in the JsonSerializerContext.
3. Check comparators for precision/culture issues (DateTimeOffset rounding, decimal, ordinal vs culture string compares).

Example fix

// before (field without stable serialization/compare)
public sealed record JournalEvent(string EventId, DateTimeOffset Timestamp, ...);

// after (explicit Ids + ordinal-aware compare)
[GenerateSerializer]
public sealed record JournalEvent(
    [property: Id(0)] string EventId,
    [property: Id(1)] DateTimeOffset Timestamp,
    ...);
// comparator uses DateTimeOffset == and StringComparison.Ordinal

static void RequireEqual<T>(string name, IReadOnlyList<T> written, IReadOnlyList<T> recovered, Func<T,T,bool> eq) {
    for (int i = 0; i < written.Count; i++)
        if (!eq(written[i], recovered[i]))
            throw new InvalidOperationException($"{name}[{i}] differs: {Serialize(written[i])} vs {Serialize(recovered[i])}");
}

When it happens

Trigger: After deactivation+recovery, for some index i, equals(written[i], recovered[i]) returns false for one of the collection comparators (InventoryEntryEquals, JournalEventEquals, WorkItemEquals, or the tags ordinal string compare). Counts matched, so this is a per-field value divergence.

Common situations: A field is not serialized/deserialized correctly (missing [Id(n)] attribute, missing source-gen for a type), a value-type equality bug (e.g., DateTimeOffset precision, decimal culture), or a journal codec dropping a sub-field. Schema drift between write and recover.

Related errors

• Recovered {name} count does not match. Written: {written.Cou
• The grain did not reactivate after DeactivateOnIdle.
• Recovered {name} does not match. Written: {Serialize(written
• Data too large to write to Azure table. Size={0} MaxSize={1}
• Unable to convert from storage format GrainStateEntity.Data=

AI-assisted analysis of dotnet/orleans@fca799fa70 (2026-08-13). Data as JSON: /api/errors/9b829f8cd97d1e95. Report an issue: GitHub.