microsoft/FASTER · error

Unexpected checkpoint status for COLD store

Error message

Unexpected checkpoint status for COLD store [expected: IDLE, actual: %s]

What it means

Before requesting the cold-store checkpoint, the driver requires cold_store_status to be IDLE. Any other value means a cold-store checkpoint is already requested, active, or finished from a previous/partial checkpoint, so starting a new one would corrupt its accounting.

Solutions

1. Wait until the previous checkpoint completes before calling Checkpoint() again
2. Serialize checkpoint initiation across threads with a mutex or atomic guard
3. If the status is stuck (e.g., after a FAILED checkpoint), reset the instance or the checkpoint state before retrying
4. Check earlier error logs (hot-store failures) that may have left the status dirty

Example fix

// before
while (busy) { f2->Checkpoint(); } // re-issues before prior completes
// after
if (checkpoint_in_flight.compare_exchange_strong(false,true)) { f2->Checkpoint(); checkpoint_in_flight.store(false); }

if (!previous_checkpoint_finished) { return Status::Aborted; }

When it happens

Trigger: Calling Checkpoint() again while a previous cold-store checkpoint request was never consumed or completed (status stuck in REQUESTED/ACTIVE/FINISHED).

Common situations: Overlapping checkpoint invocations; a prior checkpoint that failed mid-way leaving cold_store_status non-IDLE; reusing a F2 instance across checkpoint rounds without reset.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Unexpected checkpoint phase
• Unexpected checkpoint phase
• Unexpected checkpoint status
• Invalid Enum Argument
• Unexpected checkpoint phase during recovery

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/bdfe93fc57d51530. Report an issue: GitHub.

Appendix: source

Thrown at cc/src/core/f2.h:685

    // Mark hot store checkpoint status to finished (or failed)
    Status global_result = checkpoint->hot_store_checkpoint_result.load();
    log_debug("Hot store checkpoint result: %s", StatusStr(global_result));

    status = (global_result == Status::Ok) ? StoreCheckpointStatus::FINISHED : StoreCheckpointStatus::FAILED;
    checkpoint->hot_store_status.store(status);

    // Request cold store checkpointing
    CheckpointPhase phase;
    if ((phase = checkpoint->phase.load()) != CheckpointPhase::HOT_STORE_CHECKPOINT) {
      log_error("Unexpected checkpoint phase [expected: HOT_STORE_CHECKPOINT, actual: %s]",
                CHECKPOINT_PHASE_STR[static_cast<int>(phase)]);
    }
    log_debug("Moving to cold-store checkpoint phase");
    checkpoint->phase.store(CheckpointPhase::COLD_STORE_CHECKPOINT);

    if ((status = checkpoint->cold_store_status.load()) != StoreCheckpointStatus::IDLE) {
      log_error("Unexpected checkpoint status for COLD store [expected: IDLE, actual: %s]",
                STORE_CHECKPOINT_STATUS_STR[static_cast<int>(status)]);
      assert(false);
    }
    log_debug("Requesting cold store checkpoint...");
    checkpoint->cold_store_status.store(StoreCheckpointStatus::REQUESTED);
  }
}

template<class K, class V, class D, class HHI, class CHI>
inline Status F2Kv<K, V, D, HHI, CHI>::Recover(const Guid& token, uint32_t& version, std::vector<Guid>& session_ids) {
  CheckpointPhase phase = CheckpointPhase::REST;
  if (!checkpoint_.phase.compare_exchange_strong(phase, CheckpointPhase::RECOVER)) {
    log_error("Unexpected checkpoint phase during recovery [expected: REST, actual: %s]",
              CHECKPOINT_PHASE_STR[static_cast<int>(phase)]);
    return Status::Aborted;
  }

  uint32_t hot_store_version, cold_store_version;

View on GitHub (pinned to 321d872eab)