microsoft/FASTER · error · FasterException

Page read from storage failed, skipping page. Inner…

Error message

Page read from storage failed, skipping page. Inner exception:

What it means

FASTER's scan iterator loads log pages from disk asynchronously; when the page read or its continuation throws, WaitForFrameLoad catches it, marks the page as not loaded, advances the iterator to the next page boundary, and rethrows this wrapped FasterException. The library throws it so scanning code knows a specific page could not be read from storage rather than the whole scan silently stalling. The original exception text is appended as 'Inner exception: <full ToString()>'.

Solutions

Inspect the appended inner exception text to find the real IO failure and fix its cause (file present, credentials valid, disk space).
Verify the log directory/files are intact and the iterator's beginAddress matches the checkpoint the log was taken at.
Retry the scan after the transient storage failure clears; the iterator already advanced nextAddress past the bad page.
Use a fault-tolerant storage device wrapper (e.g. ReadCacheDevice/renumbering or graceful-failover device factory) if you want reads of missing pages to be tolerated.
If the log is known-healthy, report the underlying device exception; do not swallow it, as data in that page is unavailable.

Example fix

// before: scan assumes every page is readable
using var iter = fkv.Log.Scan(beginAddress, long.MaxValue, (out RecordInfo info) => info.Valid, scanBufferingMode: ScanBufferingMode.DoublePageBuffering);
while (iter.GetNext(out RecordInfo info)) { ... }
// after: catch page-read failures and skip/stop gracefully
try {
    using var iter = fkv.Log.Scan(beginAddress, long.MaxValue, (out RecordInfo info) => info.Valid, ScanBufferingMode.DoublePageBuffering);
    while (iter.GetNext(out RecordInfo info)) { ... }
} catch (FasterException ex) when (ex.Message.StartsWith("Page read from storage failed")) {
    logger.LogError(ex, "Scan hit unreadable page; aborting or restarting scan after beginAddress advanced");
}

Defensive patterns

Strategy: try-catch

Validate before calling

// before scanning, confirm the backing files/checkpoint are reachable
if (!Directory.Exists(logDir) || !File.Exists(Path.Combine(logDir, "checkpoint")))
    throw new InvalidOperationException("FASTER log storage missing; cannot start scan");
// and confirm the start address is within the log
if (startAddress < log.BeginAddress || startAddress > log.TailAddress)
    throw new InvalidOperationException($"Scan address {startAddress} outside [{log.BeginAddress},{log.TailAddress}]");

Try / catch

try {
    while (iter.GetNext(out RecordInfo info)) { /* process */ }
} catch (FasterException ex) when (ex.Message.Contains("Page read from storage failed")) {
    logger.LogError(ex, "Page read failed during scan at/after address {Address}; inspect inner exception", iter.NextAddress);
    // resume scan from iter.NextAddress or fail the job
}

Prevention

Always log the full exception text after 'Inner exception:' — the real cause is there
Use durable storage device wrappers or retry policies for cloud-backed devices
Validate checkpoint/log file integrity before long scans
Alert on disk-space and file-deletion events in the log directory
Keep iterator addresses at or above log.BeginAddress when resuming

When it happens

Trigger: Calling ScanIteratorBase.GetNext/BufferAndLoad when the underlying read from the IDevice (e.g. a LocalStorageDevice or AzureStorageDevice) fails during WaitForFrameLoad: corrupted or truncated log file, page beyond the file's end, device/credentials failure, or a checkpoint restore pointing at a deleted epoch file.

Common situations: Scanning a FASTER hybrid log whose backing files were deleted or truncated by another process; reading a log copied without its tail; transient cloud-storage (Azure blob) IO failures or expired SAS credentials mid-scan; disk full or hardware errors during a long scan.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/08ac17c35e89cb46. Report an issue: GitHub.

Appendix: source

Thrown at cs/src/core/Allocator/ScanIteratorBase.cs:226

        }

        internal abstract void AsyncReadPagesFromDeviceToFrame<TContext>(long readPageStart, int numPages, long untilAddress, TContext context, out CountdownEvent completed, long devicePageOffset = 0, IDevice device = null, IDevice objectLogDevice = null, CancellationTokenSource cts = null);

        private bool WaitForFrameLoad(long currentAddress, long currentFrame)
        {
            if (loaded[currentFrame].IsSet) return false;

            try
            {
                epoch?.Suspend();
                loaded[currentFrame].Wait(loadedCancel[currentFrame].Token); // Ensure we have completed ongoing load
            }
            catch (Exception e)
            {
                loadedPage[currentFrame] = -1;
                loadedCancel[currentFrame] = new CancellationTokenSource();
                Utility.MonotonicUpdate(ref nextAddress, (1 + (currentAddress >> logPageSizeBits)) << logPageSizeBits, out _);
                throw new FasterException("Page read from storage failed, skipping page. Inner exception: " + e.ToString());
            }
            finally
            {
                epoch?.Resume();
            }
            return true;
        }

        /// <summary>
        /// Dispose iterator
        /// </summary>
        public virtual void Dispose()
        {
            if (loaded != null)
            {
                // Wait for ongoing reads to complete/fail
                for (int i = 0; i < frameSize; i++)
                {

View on GitHub (pinned to 321d872eab)