dotnet/orleans · critical · InvalidOperationException

The journal storage consumer did not read all supplied journ

Error message

The journal storage consumer did not read all supplied journal data.

What it means

Thrown during AzureTableJournalStorage read when the consumer callback did not drain all bytes supplied to it via JournalBufferReader. After the storage layer feeds the consumer a completed buffer, it checks buffer.Length; any remaining bytes indicate the consumer stopped early, which would lose journal data. This is a contract violation by the consumer (the journaled-state machine), not a storage fault.

Source

Thrown at src/Azure/Orleans.Journaling.AzureStorage/AzureTableJournalStorage.cs:446

                    }

                    rowCount++;
                    bytes += ReadChunks(row, buffer, consumer, metadata);
                }
            }

            if (rowCount != headerState.ProviderState.RowCount || bytes != headerState.ProviderState.Length)
            {
                // A concurrent replace can delete rows of the generation being read; the caller must recover.
                throw CreateInconsistentHeaderStateException(
                    "Azure Table journal changed while reading; recovery is required.",
                    headerState.ETag);
            }

            consumer.Read(new JournalBufferReader(buffer.Reader, isCompleted: true), metadata);
            if (buffer.Length > 0)
            {
                throw new InvalidOperationException("The journal storage consumer did not read all supplied journal data.");
            }

            LogRead(_shared.Logger, bytes, Table.Name, _partitionKey);
            succeeded = true;
        }
        finally
        {
            _shared.Instruments.OnOperationCompleted(
                AzureTableJournalStorageInstruments.OperationRead,
                Stopwatch.GetElapsedTime(startTimestamp),
                bytes,
                succeeded);
        }
    }

    public async ValueTask ReplaceAsync(ReadOnlySequence<byte> value, CancellationToken cancellationToken)
    {
        var startTimestamp = Stopwatch.GetTimestamp();

View on GitHub (pinned to fca799fa70)

Solutions

Ensure the consumer reads the buffer to completion (the contract requires consuming all bytes until isCompleted is true).
Update the IJournalFormat implementation to match the version that wrote the data.
If the data is genuinely corrupt, force a compaction to rewrite the generation from the last consistent checkpoint.

Defensive patterns

Strategy: validation

Validate before calling

// Ensure your IJournalConsumer drains the buffer fully:
public void Read(JournalBufferReader reader, IReadOnlyDictionary<string,string>? metadata)
{
    while (reader.TryRead(out var record)) { /* process */ }
    // reader must be exhausted before returning; storage asserts buffer.Length == 0.

Prevention

Always drain the JournalBufferReader to completion in your consumer.
Keep the format writer and reader versions in sync.
Add an integration test that round-trips a payload and asserts no leftover bytes.

When it happens

Trigger: The IJournalConsumer.Read callback returns before consuming all bytes in the supplied JournalBufferReader, leaving buffer.Length > 0. This is an internal contract violation surfaced during ReadAsync.

Common situations: A custom IJournalFormat or IJournalConsumer with a bug that short-circuits on a partial record; version mismatch between the format writer and reader; malformed payload that causes the consumer to throw internally and be swallowed.

Related errors

AI-assisted analysis of dotnet/orleans@fca799fa70 (2026-08-13). Data as JSON: /api/errors/3e68b7b2138279c9. Report an issue: GitHub.