{"record":{"id":"bdfe93fc57d51530","repo":"microsoft/FASTER","slug":"unexpected-checkpoint-status-for-cold-store-expected-idle","errorCode":null,"errorMessage":"Unexpected checkpoint status for COLD store [expected: IDLE, actual: %s]","messagePattern":"Unexpected checkpoint status for COLD store \\[expected: IDLE, actual: (.+?)\\]","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cc/src/core/f2.h","lineNumber":685,"sourceCode":"\n    // Mark hot store checkpoint status to finished (or failed)\n    Status global_result = checkpoint->hot_store_checkpoint_result.load();\n    log_debug(\"Hot store checkpoint result: %s\", StatusStr(global_result));\n\n    status = (global_result == Status::Ok) ? StoreCheckpointStatus::FINISHED : StoreCheckpointStatus::FAILED;\n    checkpoint->hot_store_status.store(status);\n\n    // Request cold store checkpointing\n    CheckpointPhase phase;\n    if ((phase = checkpoint->phase.load()) != CheckpointPhase::HOT_STORE_CHECKPOINT) {\n      log_error(\"Unexpected checkpoint phase [expected: HOT_STORE_CHECKPOINT, actual: %s]\",\n                CHECKPOINT_PHASE_STR[static_cast<int>(phase)]);\n    }\n    log_debug(\"Moving to cold-store checkpoint phase\");\n    checkpoint->phase.store(CheckpointPhase::COLD_STORE_CHECKPOINT);\n\n    if ((status = checkpoint->cold_store_status.load()) != StoreCheckpointStatus::IDLE) {\n      log_error(\"Unexpected checkpoint status for COLD store [expected: IDLE, actual: %s]\",\n                STORE_CHECKPOINT_STATUS_STR[static_cast<int>(status)]);\n      assert(false);\n    }\n    log_debug(\"Requesting cold store checkpoint...\");\n    checkpoint->cold_store_status.store(StoreCheckpointStatus::REQUESTED);\n  }\n}\n\ntemplate<class K, class V, class D, class HHI, class CHI>\ninline Status F2Kv<K, V, D, HHI, CHI>::Recover(const Guid& token, uint32_t& version, std::vector<Guid>& session_ids) {\n  CheckpointPhase phase = CheckpointPhase::REST;\n  if (!checkpoint_.phase.compare_exchange_strong(phase, CheckpointPhase::RECOVER)) {\n    log_error(\"Unexpected checkpoint phase during recovery [expected: REST, actual: %s]\",\n              CHECKPOINT_PHASE_STR[static_cast<int>(phase)]);\n    return Status::Aborted;\n  }\n\n  uint32_t hot_store_version, cold_store_version;","sourceCodeStart":667,"sourceCodeEnd":703,"githubUrl":"https://github.com/microsoft/FASTER/blob/321d872eabda6a0345c8bd76419f89723ed864ae/cc/src/core/f2.h#L667-L703","documentation":"Before requesting the cold-store checkpoint, the driver requires cold_store_status to be IDLE. Any other value means a cold-store checkpoint is already requested, active, or finished from a previous/partial checkpoint, so starting a new one would corrupt its accounting.","triggerScenarios":"Calling Checkpoint() again while a previous cold-store checkpoint request was never consumed or completed (status stuck in REQUESTED/ACTIVE/FINISHED).","commonSituations":"Overlapping checkpoint invocations; a prior checkpoint that failed mid-way leaving cold_store_status non-IDLE; reusing a F2 instance across checkpoint rounds without reset.","solutions":["Wait until the previous checkpoint completes before calling Checkpoint() again","Serialize checkpoint initiation across threads with a mutex or atomic guard","If the status is stuck (e.g., after a FAILED checkpoint), reset the instance or the checkpoint state before retrying","Check earlier error logs (hot-store failures) that may have left the status dirty"],"exampleFix":"// before\nwhile (busy) { f2->Checkpoint(); } // re-issues before prior completes\n// after\nif (checkpoint_in_flight.compare_exchange_strong(false,true)) { f2->Checkpoint(); checkpoint_in_flight.store(false); }","handlingStrategy":"validation","validationCode":"if (!previous_checkpoint_finished) { return Status::Aborted; }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Serialize checkpoint calls","Handle FAILED checkpoints by resetting state before retry","Avoid reusing a checkpoint context across rounds"],"tags":["checkpoint","state-machine","cold-store","concurrency"],"backgroundTag":"invalid-state-transition","analyzedSha":"321d872eabda6a0345c8bd76419f89723ed864ae","analyzedAt":"2026-09-15T22:18:00.693Z","contentChangedAt":"2026-09-15T22:18:00.693Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}