{"record":{"id":"492fbd719f86d950","repo":"microsoft/FASTER","slug":"expected-empty-entry-found-it-occupied","errorCode":null,"errorMessage":"Expected empty entry -- found it occupied...","messagePattern":"Expected empty entry -- found it occupied\\.\\.\\.","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"cc/src/index/mem_index.h","lineNumber":839,"sourceCode":"        AtomicHashBucketEntry* front_atomic_entry = &(front_bucket->entries[front_entry_idx]);\n        hash_bucket_entry_t front_bucket_entry{ front_atomic_entry->load() };\n        HashBucketEntry front_bucket_expected_entry{ front_bucket_entry };\n\n        if (front_bucket_entry.unused()) {\n          if (next_free_entry == nullptr) {\n            next_free_entry = front_atomic_entry;\n            back_bucket = front_bucket;\n            back_entry_idx = front_entry_idx;\n          }\n          continue;\n        }\n        // Entry is occupied!\n\n        if (next_free_entry) {\n          // Move occupied entry to closest (forward-wise) free entry\n          HashBucketEntry empty_entry{ HashBucketEntry::kInvalidEntry };\n          if (!next_free_entry->compare_exchange_strong(empty_entry, front_bucket_expected_entry)) {\n            log_error(\"Expected empty entry -- found it occupied...\");\n          } else {\n            if (!front_atomic_entry->compare_exchange_strong(front_bucket_expected_entry, empty_entry)) {\n              log_error(\"Expected occupied entry -- found it empty...\");\n            }\n          }\n          next_free_entry = nullptr;\n\n          // Try to find next available free entry\n          AtomicHashBucketEntry* back_atomic_entry;\n          do {\n            for (; back_entry_idx < hash_bucket_t::kNumEntries; ++back_entry_idx) {\n              back_atomic_entry = &(back_bucket->entries[back_entry_idx]);\n              hash_bucket_entry_t back_bucket_entry{ back_atomic_entry->load() };\n              if (back_bucket_entry.unused()) {\n                // Found new available entry!\n                next_free_entry = back_atomic_entry;\n                break;\n              }","sourceCodeStart":821,"sourceCodeEnd":857,"githubUrl":"https://github.com/microsoft/FASTER/blob/321d872eabda6a0345c8bd76419f89723ed864ae/cc/src/index/mem_index.h#L821-L857","documentation":"During hash-index checkpoint/rename (GC / relocation of entries), FASTER moves an occupied entry into a free slot. The CAS expecting the free slot to be empty failed because another thread claimed it concurrently, so the relocation of the front bucket entry is abandoned. This is an internal race log in lock-free index maintenance, typically benign for correctness but indicates heavy concurrent index mutation.","triggerScenarios":"Concurrent Checkpoint()/GrowIndex/GC (epoch-protected entry relocation) while other threads insert/delete entries in the same hash bucket — next_free_entry's slot got occupied between finding it and the compare_exchange.","commonSituations":"High-concurrency workloads with many concurrent upserts/deletes during index checkpointing or garbage collection; long-running servers calling Checkpoint() under load.","solutions":["Retry the operation — FASTER's internal logic continues with the next free entry; if user-visible, retry Checkpoint()","Reduce concurrent mutation during checkpoint (quiesce writers or use epoch barriers) if it recurs persistently","Upgrade FASTER — relocation race handling has been refined in later versions","Check epoch protection: ensure all index-mutating calls run under proper FASTER epoch guards"],"exampleFix":"null","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"Status s = ctx->Checkpoint(token, cb);\nif (s != Status::Ok) {\n  // transient race during concurrent relocation: retry once after quiescing writers\n  quiesce_writers();\n  s = ctx->Checkpoint(token, cb);\n  resume_writers();\n}","preventionTips":["Quiesce or throttle writers while running Checkpoint()/GrowIndex","Keep index mutation within FASTER epoch-protected APIs","Expect this log under high concurrency; treat persistent occurrences as a bug report trigger","Upgrade FASTER to benefit from improved lock-free relocation"],"tags":["concurrency","hash-index","race-condition","faster"],"backgroundTag":"internal-invariant-violation","analyzedSha":"321d872eabda6a0345c8bd76419f89723ed864ae","analyzedAt":"2026-09-15T22:18:00.693Z","contentChangedAt":"2026-09-15T22:18:00.693Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}