{"record":{"id":"617ce41a45a44a47","repo":"apache/iceberg","slug":"hive-lock-heartbeat-thread-not-active","errorCode":null,"errorMessage":"Hive lock heartbeat thread not active","messagePattern":"Hive lock heartbeat thread not active","errorType":"exception","errorClass":"LockException","httpStatus":null,"severity":"error","filePath":"hive-metastore/src/main/java/org/apache/iceberg/hive/MetastoreLock.java","lineNumber":166,"sourceCode":"    // Starting heartbeat for the HMS lock\n    heartbeat = new Heartbeat(metaClients, hmsLockId.get(), lockHeartbeatIntervalTime);\n    heartbeat.schedule(exitingScheduledExecutorService);\n  }\n\n  @Override\n  public void ensureActive() throws LockException {\n    if (heartbeat == null) {\n      throw new LockException(\"Lock is not active\");\n    }\n\n    if (heartbeat.encounteredException != null) {\n      throw new LockException(\n          heartbeat.encounteredException,\n          \"Failed to heartbeat for hive lock. %s\",\n          heartbeat.encounteredException.getMessage());\n    }\n    if (!heartbeat.active()) {\n      throw new LockException(\"Hive lock heartbeat thread not active\");\n    }\n  }\n\n  @Override\n  public void unlock() {\n    if (heartbeat != null) {\n      heartbeat.cancel();\n      exitingScheduledExecutorService.shutdown();\n    }\n\n    try {\n      unlock(hmsLockId);\n    } finally {\n      releaseJvmLock();\n    }\n  }\n\n  private long acquireLock() throws LockException {","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/hive-metastore/src/main/java/org/apache/iceberg/hive/MetastoreLock.java#L148-L184","documentation":"The heartbeat thread exists but is no longer running (active() returned false), meaning the lock heartbeat stopped without recording an exception. The lock may have lapsed, so ensureActive fails to protect the commit.","triggerScenarios":"Heartbeat scheduler thread died or was shut down (executor shutdown, unhandled error killing the scheduled task) while a lock was held; ensureActive called afterwards.","commonSituations":"JVM under extreme memory pressure killing threads; executor closed prematurely; custom subclass mismanaging the ScheduledExecutorService lifecycle.","solutions":["Retry the commit with a freshly acquired lock.","Verify the heartbeat executor is not shut down while the lock is held (check CloseableGroup/executor lifecycle).","Inspect thread dumps/logs to find why the scheduled heartbeat task stopped."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  commit();\n} catch (CommitFailedException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"heartbeat thread not active\")) {\n    // re-acquire a fresh lock and retry\n  }\n}","preventionTips":["Don't shut down shared executors while locks are held.","Watch JVM health (memory/GC) so scheduled heartbeat threads survive.","Keep executor lifecycle tied to the lock lifecycle via try-with-resources/CloseableGroup."],"tags":["hive","locking","heartbeat","threading"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}