microsoft/garnet · error · TsavoriteException

Page read from storage failed, skipping page. Inner exceptio

Error message

Page read from storage failed, skipping page. Inner exception: 

What it means

ScanIteratorBase loads pages from memory/disk while iterating. If a page load throws and the exception is not OperationCanceledException, it is wrapped and rethrown as a TsavoriteException with the original exception's full ToString appended. This is the catch-all for I/O, deserialization, or device failures encountered while bringing a page into the scan frame.

Source

Thrown at libs/storage/Tsavorite/cs/src/core/Allocator/ScanIteratorBase.cs:339

                epoch?.Suspend();
                loadCompletionEvents[currentFrame].Wait(loadCTSs[currentFrame].Token); // Ensure we have completed ongoing load
            }
            catch (Exception e)
            {
                // Exception occurred so skip the page containing the currentAddress, and reinitialize the loaded page and cancellation token for the current frame.
                // The exception may have been an OperationCanceledException.
                loadedPages[currentFrame] = -1;
                loadCTSs[currentFrame] = new CancellationTokenSource();
                _ = Utility.MonotonicUpdate(ref nextAddress, GetLogicalAddressOfStartOfPage(1 + allocator.GetPageOfAddress(currentAddress, logPageSizeBits), logPageSizeBits), out _);

                // Callers may be looking for an OCE so throw that if it's what we got.
                if (e is OperationCanceledException)
                {
                    logger?.LogWarning(e, "Wait for frame load was canceled, skipping page. CurrentAddress: {currentAddress}, currentFrame: {currentFrame}", AddressString(currentAddress), currentFrame);
                    throw;
                }
                else
                    throw new TsavoriteException("Page read from storage failed, skipping page. Inner exception: " + e.ToString());
            }
            finally
            {
                epoch?.Resume();
            }
            return true;
        }

        /// <summary>
        /// Dispose iterator
        /// </summary>
        public virtual void Dispose()
        {
            // Wait for all deferred DoReadPage callbacks and their async I/O to complete before freeing
            // resources. The counter is incremented before BumpCurrentEpoch registration and decremented
            // in AsyncReadPageFromDeviceToFrameCallback when I/O completes, so reaching zero guarantees
            // no outstanding access to our state. The deferred callbacks will be drained by other threads'
            // epoch operations (Resume, Suspend, ProtectAndDrain).

View on GitHub (pinned to 951b0fc683)

Solutions

1. Inspect the inner exception text embedded in the message — it carries the real IOException/UnauthorizedAccessException/etc.
2. Verify the storage device/path still exists and is readable before scanning.
3. Reload from a known-good checkpoint and ensure it is not deleted during the scan.
4. For transient I/O (network mount hiccup), stabilize storage then restart the scan.
5. Check checkpoint version compatibility if this started after a Tsavorite upgrade.

// Before scanning, confirm the device and checkpoint are present and readable
if (!storageDevice.Exists || checkpoint is null)
    throw new InvalidOperationException("storage/checkpoint unavailable for scan");

try { while (scanIterator.GetNext(...)) { /* ... */ } }
catch (TsavoriteException ex) when (ex.Message.StartsWith("Page read from storage failed"))
{
    // ex.Message embeds the inner exception's ToString; parse/inspect it
    logger.LogError(ex, "scan aborted; inner: {Inner}", ex.InnerException);
    // decide: retry on transient I/O, or fail fast on corrupt checkpoint
}

When it happens

Trigger: A disk/memory page read fails during a scan: device read error, missing or corrupt checkpoint page, closed/disposed device, permission denied on the storage path, hash/log page returning unexpected bytes, or storage disconnected mid-scan.

Common situations: Scanning a checkpoint that was deleted or moved while the scan ran; storage on a network mount that dropped; checkpoint format/version mismatch after an upgrade; disk full or hardware fault; scanning after the device was disposed.

Related errors

• Cannot cast a DiskLogRecord to a memory LogRecord.
• RetrieveCheckpointFile: unexpected state{retStateType}
• MutablePercent must be between 10 and 95
• Store Log Memory size or PageCount must be specified
• Index size {IndexMemorySize} should not be less than index m

AI-assisted analysis of microsoft/garnet@951b0fc683 (2026-08-13). Data as JSON: /api/errors/dceaa73af0520e42. Report an issue: GitHub.