What causes "Hot store checkpoint failed!" in microsoft/FASTER?

Ensure at least one active session exists before calling Checkpoint() (attach a session first)

Hot store checkpoint failed!

Error message

Hot store checkpoint failed!

What it means

The driver attempted to issue the hot-store internal checkpoint (InternalCheckpoint) but it returned false, meaning the hot store could not start checkpointing (e.g., no active sessions, callback registration issue, or store-level failure). Status is then marked FAILED but the message indicates the hot checkpoint did not start.

Solutions

Ensure at least one active session exists before calling Checkpoint() (attach a session first)
Check the hot store is fully initialized and not being closed when the checkpoint is issued
Inspect hot_store status/logs for the underlying InternalCheckpoint failure reason
Retry the checkpoint after conditions are corrected; the status is recorded as FAILED so the next round can proceed

Example fix

// before
f2->Checkpoint(); // no active sessions
// after
if (f2->NumActiveSessions() == 0) { session = f2->NewSession(); /* attach */ }
f2->Checkpoint();

Defensive patterns

Strategy: retry

Validate before calling

if (f2_active_sessions == 0) { attach_session(); }
if (store_closing.load()) { return; }

Try / catch

bool ok = issue_checkpoint();
if (!ok) { log_error("checkpoint not started; retrying later"); schedule_retry(); }

Prevention

Ensure active sessions exist before Checkpoint()
Do not checkpoint during shutdown
Treat a false InternalCheckpoint return as retryable, not fatal

When it happens

Trigger: Calling Checkpoint() when hot_store has zero active sessions or the underlying hot store refuses to start a checkpoint (internal state busy/closed); InternalCheckpoint returning false for any store-level reason.

Common situations: Checkpoint triggered with no active client sessions attached; checkpoint issued after the store began closing; resource exhaustion preventing the checkpoint thread from starting.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/f8c2977cb08e8768. Report an issue: GitHub.

Appendix: source

Thrown at cc/src/core/f2.h:885

    cold_log_compaction_config.check_interval
  );
  bool wait;
  auto should_checkpoint_cold_store = [&cold_store_status = checkpoint_.cold_store_status]() {
    return (cold_store_status == StoreCheckpointStatus::REQUESTED);
  };
  uint64_t until_address;

  do {
    compaction_scheduled_.store(true);

    // Hot log checkpoint
    if (checkpoint_.hot_store_status == StoreCheckpointStatus::REQUESTED) {
      checkpoint_.SetNumActiveThreads(hot_store.NumActiveSessions()); // cannot be set previously due to other background ops
      log_debug("Issuing hot store checkpoint... (active sessions: %u)", hot_store.NumActiveSessions());
      bool success = hot_store.InternalCheckpoint(nullptr, F2Kv<K, V, D>::HotStoreCheckpointedCallback,
                                                  checkpoint_.token, static_cast<void*>(&checkpoint_));
      if (!success) {
        log_error("Hot store checkpoint failed!");
      }

      auto desired = success ? StoreCheckpointStatus::ACTIVE : StoreCheckpointStatus::FAILED;
      checkpoint_.hot_store_status.store(desired);
      log_debug("Hot store checkpoint issued! (status: %s)",
                STORE_CHECKPOINT_STATUS_STR[static_cast<int>(checkpoint_.hot_store_status)]);
    }

    // Cold log checkpoint (high-priority)
    if (should_checkpoint_cold_store() && checkpoint_.IsLazyCheckpointExpired()) {
      auto callback = [](void* ctxt, Status result, uint64_t persistent_serial_num) {
        auto checkpoint = static_cast<HotColdCheckpointState*>(ctxt);
        assert(checkpoint->phase.load() == CheckpointPhase::COLD_STORE_CHECKPOINT);
        assert(checkpoint->cold_store_status.load() == StoreCheckpointStatus::ACTIVE);

        StoreCheckpointStatus status;
        if ((status = checkpoint->cold_store_status.load()) != StoreCheckpointStatus::ACTIVE) {
          log_error("Unexpected checkpoint status for COLD store [expected: %s, actual: %s]",

View on GitHub (pinned to 321d872eab)