{"record":{"id":"a36d4f580ec20669","repo":"microsoft/FASTER","slug":"cold-store-checkpoint-failed","errorCode":null,"errorMessage":"Cold store checkpoint failed!","messagePattern":"Cold store checkpoint failed!","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cc/src/core/f2.h","lineNumber":919,"sourceCode":"        StoreCheckpointStatus status;\n        if ((status = checkpoint->cold_store_status.load()) != StoreCheckpointStatus::ACTIVE) {\n          log_error(\"Unexpected checkpoint status for COLD store [expected: %s, actual: %s]\",\n                    STORE_CHECKPOINT_STATUS_STR[static_cast<int>(StoreCheckpointStatus::ACTIVE)],\n                    STORE_CHECKPOINT_STATUS_STR[static_cast<int>(status)]);\n          assert(false);\n        }\n\n        // Set checkpoint status for cold-log\n        status = (result == Status::Ok) ? StoreCheckpointStatus::FINISHED : StoreCheckpointStatus::FAILED;\n        checkpoint->cold_store_status.store(status);\n        log_debug(\"Cold store checkpoint result: %s\", StatusStr(result));\n      };\n\n      log_debug(\"Issuing cold store checkpoint... (active sessions: %u)\", cold_store.NumActiveSessions());\n      bool success = cold_store.InternalCheckpoint(nullptr, callback, checkpoint_.token,\n                                                  static_cast<void*>(&checkpoint_));\n      if (!success) {\n        log_error(\"Cold store checkpoint failed!\");\n      }\n\n      auto desired = success ? StoreCheckpointStatus::ACTIVE : StoreCheckpointStatus::FAILED;\n      checkpoint_.cold_store_status.store(desired);\n      log_debug(\"Cold store checkpoint issued! (status: %s)\",\n                STORE_CHECKPOINT_STATUS_STR[static_cast<int>(checkpoint_.cold_store_status)]);\n    }\n\n    // Hot-Cold compaction (optional cold-log checkpoint)\n    if (ShouldCompactHlog(hot_store, StoreType::HOT, hot_log_compaction_config, until_address)) {\n      hot_store.StartSession();\n      cold_store.StartSession();\n      if (!CompactLog(hot_store, StoreType::HOT, until_address, true,\n                      hot_log_compaction_config.num_threads, should_checkpoint_cold_store())) {\n        log_error(\"Compact HOT log failed! :(\");\n      }\n      cold_store.StopSession();\n      hot_store.StopSession();","sourceCodeStart":901,"sourceCodeEnd":937,"githubUrl":"https://github.com/microsoft/FASTER/blob/321d872eabda6a0345c8bd76419f89723ed864ae/cc/src/core/f2.h#L901-L937","documentation":"Issuing a checkpoint on the cold store via InternalCheckpoint returned false, meaning the checkpoint could not be started (e.g. no active session on that store, a checkpoint already in progress, or internal init failure). The library logs this and marks the cold store checkpoint status as FAILED; the overall checkpoint will not include a valid cold-store snapshot.","triggerScenarios":"Calling Checkpoint()/CompactLog(with checkpoint) when the cold store has no active session at issue time, a checkpoint is already in progress (checkpoint_.token reuse), or InternalCheckpoint fails to allocate/register the checkpoint state.","commonSituations":"Calling Checkpoint() from a thread that never called StartSession(); issuing checkpoints back-to-back without waiting for the previous one to complete; hybrid-log hot/cold setups after a prior failed checkpoint left stale state.","solutions":["Call StartSession() on the cold store (and hot store) in the thread that issues the checkpoint, and StopSession() afterwards.","Wait for the previous checkpoint to complete (poll CheckpointState / wait for status != ACTIVE/REQUESTED) before issuing a new one.","Check preceding debug log 'Issuing cold store checkpoint...' for active session count of 0 and add the missing session.","Retry the checkpoint once status is FAILED; if it persists, restart with a fresh store to clear stale checkpoint state."],"exampleFix":"// before\nbool ok = kv.Checkpoint();\n// after\ncold_store.StartSession();\nbool ok = kv.Checkpoint();\ncold_store.StopSession();\nif (ok) { /* wait for checkpoint completion */ }","handlingStrategy":"retry","validationCode":"// ensure no checkpoint is pending before issuing\nif (kv.CheckpointStatus().phase != CheckpointPhase::INVALID) { /* wait */ }\nthread_local FasterKv::Session* s = ...; // thread must hold an active session","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Issue checkpoints only from threads with an active session.","Wait for one checkpoint to finish before starting another.","Poll CheckpointState instead of fire-and-forget Checkpoint().","Treat a single FAILED as retryable; persistently FAILED points to stale state needing a restart."],"tags":["checkpoint","cold-store","session","hybridlog"],"backgroundTag":"invalid-state-transition","analyzedSha":"321d872eabda6a0345c8bd76419f89723ed864ae","analyzedAt":"2026-09-15T22:18:00.693Z","contentChangedAt":"2026-09-15T22:18:00.693Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}