{"record":{"id":"864c6de900287b5d","repo":"microsoft/FASTER","slug":"expected-occupied-entry-found-it-empty","errorCode":null,"errorMessage":"Expected occupied entry -- found it empty...","messagePattern":"Expected occupied entry -- found it empty\\.\\.\\.","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"cc/src/index/mem_index.h","lineNumber":842,"sourceCode":"\n        if (front_bucket_entry.unused()) {\n          if (next_free_entry == nullptr) {\n            next_free_entry = front_atomic_entry;\n            back_bucket = front_bucket;\n            back_entry_idx = front_entry_idx;\n          }\n          continue;\n        }\n        // Entry is occupied!\n\n        if (next_free_entry) {\n          // Move occupied entry to closest (forward-wise) free entry\n          HashBucketEntry empty_entry{ HashBucketEntry::kInvalidEntry };\n          if (!next_free_entry->compare_exchange_strong(empty_entry, front_bucket_expected_entry)) {\n            log_error(\"Expected empty entry -- found it occupied...\");\n          } else {\n            if (!front_atomic_entry->compare_exchange_strong(front_bucket_expected_entry, empty_entry)) {\n              log_error(\"Expected occupied entry -- found it empty...\");\n            }\n          }\n          next_free_entry = nullptr;\n\n          // Try to find next available free entry\n          AtomicHashBucketEntry* back_atomic_entry;\n          do {\n            for (; back_entry_idx < hash_bucket_t::kNumEntries; ++back_entry_idx) {\n              back_atomic_entry = &(back_bucket->entries[back_entry_idx]);\n              hash_bucket_entry_t back_bucket_entry{ back_atomic_entry->load() };\n              if (back_bucket_entry.unused()) {\n                // Found new available entry!\n                next_free_entry = back_atomic_entry;\n                break;\n              }\n              if (back_atomic_entry == front_atomic_entry) {\n                break; // Reached front pointer!\n              }","sourceCodeStart":824,"sourceCodeEnd":860,"githubUrl":"https://github.com/microsoft/FASTER/blob/321d872eabda6a0345c8bd76419f89723ed864ae/cc/src/index/mem_index.h#L824-L860","documentation":"The mirror-image failure of the relocation step: after moving the occupied entry into a free slot succeeded, FASTER tries to clear the original front entry via CAS but the entry was already changed by a concurrent thread (expected it occupied, found it empty/different). The relocation transaction is effectively aborted; internal logic continues scanning for another free entry.","triggerScenarios":"Concurrent deletion or relocation touching the same front bucket entry between the first successful CAS and the clearing CAS, during Checkpoint()/GC/GrowIndex under concurrent load.","commonSituations":"Same as the companion 'found it occupied' race: high thread counts doing RMW/delete on hot keys while index maintenance (checkpoint/grow) runs.","solutions":["Retry — the algorithm continues; if failures correlate with user-visible issues, retry the user operation","Serialize heavy maintenance (Checkpoint/GrowIndex) away from peak write concurrency","Upgrade FASTER to a version with improved concurrent relocation handling","Ensure all index operations go through FASTER's epoch-protected APIs, not raw bucket manipulation"],"exampleFix":"null","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"Status s = ctx->Checkpoint(token, cb);\nif (s != Status::Ok) {\n  quiesce_writers();\n  s = ctx->Checkpoint(token, cb);\n  resume_writers();\n}","preventionTips":["Run index maintenance (checkpoint/grow/GC) during low-write windows","Use epoch barriers around maintenance to reduce CAS races","Treat one-off occurrences as benign; investigate only if they recur with correctness symptoms","Keep FASTER version current for improved concurrent relocation logic"],"tags":["concurrency","hash-index","race-condition","faster"],"backgroundTag":"internal-invariant-violation","analyzedSha":"321d872eabda6a0345c8bd76419f89723ed864ae","analyzedAt":"2026-09-15T22:18:00.693Z","contentChangedAt":"2026-09-15T22:18:00.693Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}