{"record":{"id":"ccb2bfc1d564cb91","repo":"dotnet/orleans","slug":"azure-table-journal-batch-of-length-n0-bytes-exc","errorCode":null,"errorMessage":"Azure Table journal batch of {length:N0} bytes exceeds the per-transaction limit of {MaxAppendBytes:N0} bytes (2 MiB). Reduce the operation size or compact more aggressively.","messagePattern":"Azure Table journal batch of (.+?) bytes exceeds the per-transaction limit of (.+?) bytes \\(2 MiB\\)\\. Reduce the operation size or compact more aggressively\\.","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"error","filePath":"src/Azure/Orleans.Journaling.AzureStorage/AzureTableJournalStorage.cs","lineNumber":570,"sourceCode":"        finally\n        {\n            _shared.Instruments.OnOperationCompleted(\n                AzureTableJournalStorageInstruments.OperationReplace,\n                Stopwatch.GetElapsedTime(startTimestamp),\n                value.Length,\n                succeeded);\n        }\n    }\n\n    private static void ThrowIfBatchTooLarge(long length)\n    {\n        // Azure rejects oversize entity group transactions, so fail locally with the journal-specific guidance.\n        if (length <= MaxAppendBytes)\n        {\n            return;\n        }\n\n        throw new InvalidOperationException(\n            $\"Azure Table journal batch of {length:N0} bytes exceeds the per-transaction limit of {MaxAppendBytes:N0} bytes (2 MiB). \" +\n            \"Reduce the operation size or compact more aggressively.\");\n    }\n\n    private async ValueTask EnsureHeaderAsync(CancellationToken cancellationToken)\n    {\n        // Either create the initial header or load the header created by a racing instance, then loop until state is cached.\n        while (!HeaderExists)\n        {\n            try\n            {\n                var created = await CreateHeaderAsync(callerMetadata: null, cancellationToken).ConfigureAwait(false);\n                if (created.ETag != default)\n                {\n                    SetHeader(created.ETag, created.ProviderState);\n                    return;\n                }\n","sourceCodeStart":552,"sourceCodeEnd":588,"githubUrl":"https://github.com/dotnet/orleans/blob/fca799fa70ecb6ad975224271703ca43221f58de/src/Azure/Orleans.Journaling.AzureStorage/AzureTableJournalStorage.cs#L552-L588","documentation":"Thrown by AzureTableJournalStorage.ThrowIfBatchTooLarge when an Append or Replace operation would produce an entity group transaction exceeding 2 MiB (MaxAppendBytes). Azure Table entity group transactions are capped at 4 MiB; the journal reserves headroom for Base64 encoding and overhead, so it fails locally at 2 MiB with actionable guidance rather than letting Azure reject it opaquely.","triggerScenarios":"Calling AppendAsync with a payload that, after chunking into 64 KiB table properties and adding the header fence entity, exceeds MaxAppendBytes (2 MiB). Or calling ReplaceAsync with a snapshot large enough to exceed the limit in a single transaction.","commonSituations":"Appending a large batch in one call without pre-chunking; insufficient compaction so accumulated rows push a replace over the limit; misjudging payload size after encoding.","solutions":["Split the append into smaller batches each under ~1.5 MiB to stay clear of the 2 MiB cap.","Trigger compaction (ReplaceAsync with a checkpoint) more frequently so generations stay small.","If using ReplaceAsync for a large snapshot, ensure the snapshot itself is under 2 MiB or implement a custom chunking strategy."],"exampleFix":"// before\nawait storage.AppendAsync(largeBuffer, metadata, ct); // largeBuffer > 2 MiB\n\n// after\nforeach (var chunk in SplitIntoChunks(largeBuffer, chunkSize: 1_500_000))\n{\n    await storage.AppendAsync(chunk, metadata, ct);\n}","handlingStrategy":"validation","validationCode":"const long SafeAppendCeiling = 1_500_000; // 1.5 MiB headroom under 2 MiB cap\nforeach (var chunk in SplitIntoChunks(value, SafeAppendCeiling))\n{\n    await storage.AppendAsync(chunk, metadata, ct);\n}\n\nstatic IEnumerable<ReadOnlyMemory<byte>> SplitIntoChunks(ReadOnlyMemory<byte> data, long max)\n{\n    for (var i = 0; i < data.Length; i += (int)max)\n        yield return data.Slice(i, (int)Math.Min(max, data.Length - i));\n}","typeGuard":null,"tryCatchPattern":"try { await storage.AppendAsync(value, metadata, ct); }\ncatch (InvalidOperationException ex) when (ex.Message.Contains(\"exceeds the per-transaction limit\"))\n{\n    logger.LogWarning(\"Append too large; splitting into smaller batches.\");\n    foreach (var chunk in SplitIntoChunks(value, 1_500_000))\n        await storage.AppendAsync(chunk, metadata, ct);\n}","preventionTips":["Chunk large appends client-side to stay under 1.5 MiB per call.","Compact frequently so generations remain small.","Monitor append payload sizes and alert near the 2 MiB ceiling."],"tags":["azure-table-storage","size-limit","append","invalid-operation"],"backgroundTag":null,"analyzedSha":"fca799fa70ecb6ad975224271703ca43221f58de","analyzedAt":"2026-08-13T19:55:57.938Z","schemaVersion":2},"datasetVersion":"2026-08-14T00:17:13.853Z"}