{"record":{"id":"0dd75f0335c8baaf","repo":"microsoft/FASTER","slug":"wrong-amount-of-data-received-from-page-blob-expected-length","errorCode":null,"errorMessage":"wrong amount of data received from page blob, expected={length}, actual={stream.Position}","messagePattern":"wrong amount of data received from page blob, expected=(.+?), actual=(.+?)","errorType":"exception","errorClass":"InvalidDataException","httpStatus":null,"severity":"error","filePath":"cs/src/devices/AzureStorageDevice/AzureStorageDevice.cs","lineNumber":606,"sourceCode":"                            }\n\n                            if (length > 0)\n                            {\n                                var client = (numAttempts > 1 || length == MAX_DOWNLOAD_SIZE) ? blob.Default : blob.Aggressive;\n\n                                var response = await client.DownloadStreamingAsync(\n                                    range: new Azure.HttpRange(sourceAddress + offset, length),\n                                    conditions: null,\n                                    rangeGetContentHash: false,\n                                    cancellationToken: this.StorageErrorHandler.Token)\n                                    .ConfigureAwait(false);\n\n                                await response.Value.Content.CopyToAsync(stream).ConfigureAwait(false);\n                            }\n\n                            if (stream.Position != offset + length)\n                            {\n                                throw new InvalidDataException($\"wrong amount of data received from page blob, expected={length}, actual={stream.Position}\");\n                            }\n\n                            return length;\n                        });\n\n                    readLength -= length;\n                    offset += length;\n                }\n            }\n        }\n\n        void TryWriteAsync(BlobEntry blobEntry, IntPtr sourceAddress, ulong destinationAddress, uint numBytesToWrite, long id)\n        {\n            // If pageBlob is null, it is being created. Attempt to queue the write for the creator to complete after it is done\n            if (blobEntry.PageBlob.Default == null\n                && blobEntry.TryQueueAction(() => this.WriteToBlobAsync(blobEntry, sourceAddress, destinationAddress, numBytesToWrite, id)))\n            {\n                return;","sourceCodeStart":588,"sourceCodeEnd":624,"githubUrl":"https://github.com/microsoft/FASTER/blob/321d872eabda6a0345c8bd76419f89723ed864ae/cs/src/devices/AzureStorageDevice/AzureStorageDevice.cs#L588-L624","documentation":"AzureStorageDevice reads a range of bytes from an Azure page blob into a stream and validates that the stream received exactly offset+length bytes. This InvalidDataException is thrown when the copy from the HTTP response completed with fewer (or more) bytes than requested, indicating truncated or corrupted blob data. The library throws it because silently returning short reads would corrupt log replay.","triggerScenarios":"Calling the page-blob read path (used by Faster's Azure storage device during checkpoint/log reads) when the blob was resized/truncated, the blob was deleted or leased by another process mid-read, the requested [offset, offset+length) range extends past the blob's committed length, or a transient network failure cut the response body short.","commonSituations":"Reading a FASTER checkpoint whose blob was truncated by a failed upload; a container/blob deleted by lifecycle policy while the device was running; requesting a byte range beyond the blob size after an improper shutdown; intermittent Azure Storage network failures.","solutions":["Verify the blob exists and is at least offset+length bytes before reading; restore from backup if truncated.","Check that no other process is deleting/truncating blobs (lifecycle policies, competing instances) and that the container is not being emptied.","Retry the read on transient network failures; ensure Azure Storage retry policy is enabled.","Confirm the offset/length passed to the read matches the checkpoint metadata written earlier."],"exampleFix":"// before: reading a range that may exceed blob length\nlong length = endOffset - offset; // may exceed blob size\nawait device.ReadAsync(offset, buffer, length);\n// after: clamp the read to the actual blob size\nvar props = await blob.GetPropertiesAsync();\nlong length = Math.Min(endOffset, props.Value.ContentLength) - offset;\nawait device.ReadAsync(offset, buffer, length);","handlingStrategy":"retry","validationCode":"var props = await blob.GetPropertiesAsync();\nif (props.Value.ContentLength < offset + length)\n    throw new InvalidOperationException($\"blob too short: need {offset + length}, have {props.Value.ContentLength}\");","typeGuard":"static bool IsBlobReadShort(Stream s, long offset, long length) => s.Position != offset + length;","tryCatchPattern":"try\n{\n    await device.ReadAsync(offset, buffer, length);\n}\ncatch (InvalidDataException ex) when (ex.Message.StartsWith(\"wrong amount of data received from page blob\"))\n{\n    // log, verify blob integrity, optionally retry or restore from checkpoint\n}","preventionTips":["Validate blob length against expected checkpoint size before reading.","Ensure Azure Storage retry policy is enabled for transient network faults.","Prevent competing processes/lifecycle rules from deleting or truncating blobs.","Persist and verify checkpoint metadata (lengths) after every checkpoint."],"tags":["azure-storage","io","data-integrity"],"backgroundTag":"unexpected-response-shape","analyzedSha":"321d872eabda6a0345c8bd76419f89723ed864ae","analyzedAt":"2026-09-15T22:18:00.693Z","contentChangedAt":"2026-09-15T22:18:00.693Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}