microsoft/FASTER · error

Cold store checkpoint failed!

Error message

Cold store checkpoint failed!

What it means

Issuing a checkpoint on the cold store via InternalCheckpoint returned false, meaning the checkpoint could not be started (e.g. no active session on that store, a checkpoint already in progress, or internal init failure). The library logs this and marks the cold store checkpoint status as FAILED; the overall checkpoint will not include a valid cold-store snapshot.

Solutions

1. Call StartSession() on the cold store (and hot store) in the thread that issues the checkpoint, and StopSession() afterwards.
2. Wait for the previous checkpoint to complete (poll CheckpointState / wait for status != ACTIVE/REQUESTED) before issuing a new one.
3. Check preceding debug log 'Issuing cold store checkpoint...' for active session count of 0 and add the missing session.
4. Retry the checkpoint once status is FAILED; if it persists, restart with a fresh store to clear stale checkpoint state.

Example fix

// before
bool ok = kv.Checkpoint();
// after
cold_store.StartSession();
bool ok = kv.Checkpoint();
cold_store.StopSession();
if (ok) { /* wait for checkpoint completion */ }

// ensure no checkpoint is pending before issuing
if (kv.CheckpointStatus().phase != CheckpointPhase::INVALID) { /* wait */ }
thread_local FasterKv::Session* s = ...; // thread must hold an active session

When it happens

Trigger: Calling Checkpoint()/CompactLog(with checkpoint) when the cold store has no active session at issue time, a checkpoint is already in progress (checkpoint_.token reuse), or InternalCheckpoint fails to allocate/register the checkpoint state.

Common situations: Calling Checkpoint() from a thread that never called StartSession(); issuing checkpoints back-to-back without waiting for the previous one to complete; hybrid-log hot/cold setups after a prior failed checkpoint left stale state.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Compact COLD log failed! :(
• Unable to find valid HybridLog token
• Unexpected checkpoint status for COLD store
• Unexpected checkpoint status
• Unexpected checkpoint status for COLD store

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/a36d4f580ec20669. Report an issue: GitHub.

Appendix: source

Thrown at cc/src/core/f2.h:919

        StoreCheckpointStatus status;
        if ((status = checkpoint->cold_store_status.load()) != StoreCheckpointStatus::ACTIVE) {
          log_error("Unexpected checkpoint status for COLD store [expected: %s, actual: %s]",
                    STORE_CHECKPOINT_STATUS_STR[static_cast<int>(StoreCheckpointStatus::ACTIVE)],
                    STORE_CHECKPOINT_STATUS_STR[static_cast<int>(status)]);
          assert(false);
        }

        // Set checkpoint status for cold-log
        status = (result == Status::Ok) ? StoreCheckpointStatus::FINISHED : StoreCheckpointStatus::FAILED;
        checkpoint->cold_store_status.store(status);
        log_debug("Cold store checkpoint result: %s", StatusStr(result));
      };

      log_debug("Issuing cold store checkpoint... (active sessions: %u)", cold_store.NumActiveSessions());
      bool success = cold_store.InternalCheckpoint(nullptr, callback, checkpoint_.token,
                                                  static_cast<void*>(&checkpoint_));
      if (!success) {
        log_error("Cold store checkpoint failed!");
      }

      auto desired = success ? StoreCheckpointStatus::ACTIVE : StoreCheckpointStatus::FAILED;
      checkpoint_.cold_store_status.store(desired);
      log_debug("Cold store checkpoint issued! (status: %s)",
                STORE_CHECKPOINT_STATUS_STR[static_cast<int>(checkpoint_.cold_store_status)]);
    }

    // Hot-Cold compaction (optional cold-log checkpoint)
    if (ShouldCompactHlog(hot_store, StoreType::HOT, hot_log_compaction_config, until_address)) {
      hot_store.StartSession();
      cold_store.StartSession();
      if (!CompactLog(hot_store, StoreType::HOT, until_address, true,
                      hot_log_compaction_config.num_threads, should_checkpoint_cold_store())) {
        log_error("Compact HOT log failed! :(");
      }
      cold_store.StopSession();
      hot_store.StopSession();

View on GitHub (pinned to 321d872eab)