{"record":{"id":"a9956f2e88cc3465","repo":"apache/iceberg","slug":"cannot-heartbeat-to-a-deleted-lock","errorCode":null,"errorMessage":"Cannot heartbeat to a deleted lock ","messagePattern":"Cannot heartbeat to a deleted lock ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"warning","filePath":"core/src/main/java/org/apache/iceberg/util/LockManagers.java","lineNumber":225,"sourceCode":"\n      if (succeed) {\n        // cleanup old heartbeat\n        if (HEARTBEATS.containsKey(entityId)) {\n          HEARTBEATS.remove(entityId).cancel(false);\n        }\n\n        HEARTBEATS.put(\n            entityId,\n            scheduler()\n                .scheduleAtFixedRate(\n                    () -> {\n                      InMemoryLockContent lastContent = LOCKS.get(entityId);\n                      try {\n                        long newExpiration = System.currentTimeMillis() + heartbeatTimeoutMs();\n                        LOCKS.replace(\n                            entityId, lastContent, new InMemoryLockContent(ownerId, newExpiration));\n                      } catch (NullPointerException e) {\n                        throw new RuntimeException(\n                            \"Cannot heartbeat to a deleted lock \" + entityId, e);\n                      }\n                    },\n                    0,\n                    heartbeatIntervalMs(),\n                    TimeUnit.MILLISECONDS));\n\n      } else {\n        throw new IllegalStateException(\"Unable to acquire lock \" + entityId);\n      }\n    }\n\n    @Override\n    public boolean acquire(String entityId, String ownerId) {\n      try {\n        Tasks.foreach(entityId)\n            .retry(Integer.MAX_VALUE - 1)\n            .onlyRetryOn(IllegalStateException.class)","sourceCodeStart":207,"sourceCodeEnd":243,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/util/LockManagers.java#L207-L243","documentation":"The InMemoryLockManager heartbeat task re-reads the lock content and uses LOCKS.replace(entityId, lastContent, newContent); if the lock was removed concurrently, lastContent may be null and replace() throws NullPointerException, which is rethrown as RuntimeException 'Cannot heartbeat to a deleted lock <entityId>'. It detects heartbeat racing with lock release/unlock.","triggerScenarios":"The heartbeat executor fires after the lock entry has been removed from the LOCKS map — e.g., lock released or expired and cleaned up while the scheduled heartbeat is still running.","commonSituations":"Close/shutdown racing with heartbeats; unlock followed immediately by JVM exit while the scheduled executor is mid-tick; very short heartbeat intervals amplifying the race window.","solutions":["Ensure the lock is kept alive (heartbeated) for as long as it is used, and release it only after closing the operation","Shut down / cancel the heartbeat future before removing the lock entry","Upgrade Iceberg — recent versions guard against this heartbeat/unlock race","Catch the RuntimeException and treat it as lock loss; re-acquire before continuing"],"exampleFix":"// before\nlocks.remove(entityId); // heartbeat may still run\n// after\nheartbeatFuture.cancel(false);\nlocks.remove(entityId);","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { lockManager.acquire(entityId, ownerId); } catch (RuntimeException e) { if (e.getMessage().startsWith(\"Cannot heartbeat to a deleted lock\")) { lockManager.acquire(entityId, ownerId); } else throw e; }","preventionTips":["Cancel heartbeat futures before releasing locks","Keep lock lifetime >= operation lifetime","Upgrade to a version with the heartbeat/unlock race fixed","Catch and re-acquire rather than propagating heartbeat failures"],"tags":["lock","concurrency","race-condition","heartbeat"],"backgroundTag":"lock-contention","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}