microsoft/FASTER · critical

Error: %u

Error message

Error: %u

What it means

This log_error fires inside the page-recovery read callback of PersistentMemoryMalloc::Recover (async read of a previously flushed page back into memory). If the async read completes with a Status other than Status::Ok, the status is logged, but the callback then overwrites the page status to ReadDone anyway, so recovery may proceed with a page whose contents were never successfully read — leading to corrupted or missing log data after restart.

Solutions

  1. Map the logged status code to the underlying read failure and verify the checkpoint/log files exist, are complete, and are readable
  2. Recovery must be pointed at a valid, fully-flushed checkpoint token; list available checkpoints (e.g. via the checkpoint manager) and retry with the correct one
  3. If files are corrupt, restore from an earlier healthy checkpoint or backup rather than trusting the partial recovery
  4. Fix storage health/permissions issues and retry Recover before allowing the application to serve data
Defensive patterns

Strategy: validation

Validate before calling

// before calling Recover, confirm the checkpoint files exist
for (const std::string& f : required_checkpoint_files) {
  if (!std::filesystem::exists(checkpoint_dir / f) || std::filesystem::file_size(checkpoint_dir / f) == 0) {
    // abort recovery: incomplete checkpoint, fall back to an earlier token
  }
}

Try / catch

try {
  store->Recover(index_token, hybrid_log_token, dir_token);
} catch (...) {
  // NOTE: failed page reads are only logged ('Error: %u'), not thrown —
  // scan logs for non-Ok statuses and treat any occurrence as failed recovery:
  std::cerr << "Recovery saw failed page reads; restore from an earlier checkpoint\n";
}

Prevention

When it happens

Trigger: Calling store->Recover() (or allocator Recover) after a restart/checkpoint, where the async read of a flushed page from the checkpoint/log file fails: missing or truncated log file, file offsets beyond EOF, device I/O error, or a checkpoint directory that does not match the requested checkpoint token.

Common situations: Recovering from a checkpoint directory that was copied incompletely or corrupted; disk failure or changed volume between checkpoint and recovery; pointing recovery at the wrong checkpoint token so required pages don't exist.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors


AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/c9b4134e32d5d148. Report an issue: GitHub.

Appendix: source

Thrown at cc/src/core/persistent_memory_malloc.h:999

    Context(std::atomic<PageRecoveryStatus>& page_status_)
      : page_status{ &page_status_ } {
    }
    /// The deep-copy constructor
    Context(const Context& other)
      : page_status{ other.page_status } {
    }
   protected:
    Status DeepCopy_Internal(IAsyncContext*& context_copy) final {
      return IAsyncContext::DeepCopy_Internal(*this, context_copy);
    }
   public:
    std::atomic<PageRecoveryStatus>* page_status;
  };

  auto callback = [](IAsyncContext* ctxt, Status result, size_t bytes_transferred) {
    CallbackContext<Context> context{ ctxt };
    if(result != Status::Ok) {
      log_error("Error: %u\n", static_cast<uint8_t>(result));
    }
    assert(context->page_status->load() == PageRecoveryStatus::IssuedRead);
    context->page_status->store(PageRecoveryStatus::ReadDone);
  };

  for(uint32_t read_page = start_page; read_page < start_page + num_pages; ++read_page) {
    if(!Page(read_page)) {
      // Allocate a new page.
      AllocatePage(read_page);
    } else {
      // Clear an old used page.
      std::memset(Page(read_page), 0, kPageSize);
    }
    assert(recovery_status.page_status(read_page) == PageRecoveryStatus::NotStarted);
    recovery_status.page_status(read_page).store(PageRecoveryStatus::IssuedRead);
    PageStatus(read_page).LastFlushedUntilAddress.store(Address{ read_page + 1, 0 });
    Context context{ recovery_status.page_status(read_page) };
    RETURN_NOT_OK(read_file.ReadAsync(kPageSize * (read_page - file_start_page), Page(read_page),

View on GitHub (pinned to 321d872eab)