microsoft/FASTER · critical
Error: %u
Error message
Error: %u
What it means
This log_error fires inside the page-recovery read callback of PersistentMemoryMalloc::Recover (async read of a previously flushed page back into memory). If the async read completes with a Status other than Status::Ok, the status is logged, but the callback then overwrites the page status to ReadDone anyway, so recovery may proceed with a page whose contents were never successfully read — leading to corrupted or missing log data after restart.
Solutions
- Map the logged status code to the underlying read failure and verify the checkpoint/log files exist, are complete, and are readable
- Recovery must be pointed at a valid, fully-flushed checkpoint token; list available checkpoints (e.g. via the checkpoint manager) and retry with the correct one
- If files are corrupt, restore from an earlier healthy checkpoint or backup rather than trusting the partial recovery
- Fix storage health/permissions issues and retry Recover before allowing the application to serve data
Defensive patterns
Strategy: validation
Validate before calling
// before calling Recover, confirm the checkpoint files exist
for (const std::string& f : required_checkpoint_files) {
if (!std::filesystem::exists(checkpoint_dir / f) || std::filesystem::file_size(checkpoint_dir / f) == 0) {
// abort recovery: incomplete checkpoint, fall back to an earlier token
}
} Try / catch
try {
store->Recover(index_token, hybrid_log_token, dir_token);
} catch (...) {
// NOTE: failed page reads are only logged ('Error: %u'), not thrown —
// scan logs for non-Ok statuses and treat any occurrence as failed recovery:
std::cerr << "Recovery saw failed page reads; restore from an earlier checkpoint\n";
} Prevention
- Never point Recover at a partially copied or hand-edited checkpoint directory
- After any failed page-read log during recovery, re-verify or restore from a known-good checkpoint — ReadDone is set even on failure
- Take and retain multiple checkpoint generations so a corrupt one can be abandoned
- Ensure the checkpoint volume is stable (same device path, mounted, healthy) between checkpoint and recovery
When it happens
Trigger: Calling store->Recover() (or allocator Recover) after a restart/checkpoint, where the async read of a flushed page from the checkpoint/log file fails: missing or truncated log file, file offsets beyond EOF, device I/O error, or a checkpoint directory that does not match the requested checkpoint token.
Common situations: Recovering from a checkpoint directory that was copied incompletely or corrupted; disk failure or changed volume between checkpoint and recovery; pointing recovery at the wrong checkpoint token so required pages don't exist.
Understand the failure class
Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.
Related errors
- Unable to set first valid segment to
- Unable to set last valid segment to
- Invalid checkpoint version
- Invalid checksum for checkpoint
- Invalid log commit metadata for ID
AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15).
Data as JSON: /api/errors/c9b4134e32d5d148.
Report an issue: GitHub.
Appendix: source
Thrown at cc/src/core/persistent_memory_malloc.h:999
Context(std::atomic<PageRecoveryStatus>& page_status_)
: page_status{ &page_status_ } {
}
/// The deep-copy constructor
Context(const Context& other)
: page_status{ other.page_status } {
}
protected:
Status DeepCopy_Internal(IAsyncContext*& context_copy) final {
return IAsyncContext::DeepCopy_Internal(*this, context_copy);
}
public:
std::atomic<PageRecoveryStatus>* page_status;
};
auto callback = [](IAsyncContext* ctxt, Status result, size_t bytes_transferred) {
CallbackContext<Context> context{ ctxt };
if(result != Status::Ok) {
log_error("Error: %u\n", static_cast<uint8_t>(result));
}
assert(context->page_status->load() == PageRecoveryStatus::IssuedRead);
context->page_status->store(PageRecoveryStatus::ReadDone);
};
for(uint32_t read_page = start_page; read_page < start_page + num_pages; ++read_page) {
if(!Page(read_page)) {
// Allocate a new page.
AllocatePage(read_page);
} else {
// Clear an old used page.
std::memset(Page(read_page), 0, kPageSize);
}
assert(recovery_status.page_status(read_page) == PageRecoveryStatus::NotStarted);
recovery_status.page_status(read_page).store(PageRecoveryStatus::IssuedRead);
PageStatus(read_page).LastFlushedUntilAddress.store(Address{ read_page + 1, 0 });
Context context{ recovery_status.page_status(read_page) };
RETURN_NOT_OK(read_file.ReadAsync(kPageSize * (read_page - file_start_page), Page(read_page),View on GitHub (pinned to 321d872eab)