apache/iceberg · critical · CommitStateUnknownException

Failed to heartbeat for hive lock while committing changes…

Error message

Failed to heartbeat for hive lock while committing changes. This can lead to a concurrent commit attempt be able to overwrite this commit. Please check the commit history. If you are running into this issue, try reducing iceberg.hive.lock-heartbeat-interval-ms.

What it means

While committing, the periodic heartbeat that keeps the acquired Hive lock alive failed (LockException), so the lock may have expired and another committer could overwrite this commit. The commit's outcome is genuinely unknown, so a CommitStateUnknownException is thrown instead of a retriable CommitFailedException.

Solutions

Manually inspect the table's metadata_location / commit history to determine whether the commit landed before retrying
Increase iceberg.hive.lock-heartbeat-interval-ms headroom relative to commit duration and fix HMS connectivity
Ensure the lock's heartbeat thread is not starved (adequate client threads, no long GC pauses)
Use a metastore with reliable transactional locking (not embedded Derby)

Example fix

// before
conf.set("iceberg.hive.lock-heartbeat-interval-ms", "3000"); // too aggressive
// after
conf.set("iceberg.hive.lock-heartbeat-interval-ms", "60000"); // tolerate long commits

Defensive patterns

Strategy: try-catch

Validate before calling

// verify lock health before commit
HiveLock lock = ...; if (lock == null || !lock.isActive()) { reAcquireLock(); }

Try / catch

try {
  table.newAppend().appendFile(f).commit();
} catch (CommitStateUnknownException e) {
  // do NOT blindly retry; check whether commit landed
  Table refreshed = catalog.loadTable(ident);
  if (!alreadyContains(refreshed, f)) { refreshed.newAppend().appendFile(f).commit(); }
}

Prevention

Set a generous iceberg.hive.lock-heartbeat-interval-ms
Use a reliable remote HMS, not an embedded one
Monitor GC pauses and network stability toward HMS during writes
After CommitStateUnknownException, always inspect commit history before re-committing

When it happens

Trigger: doCommit acquires a Hive lock (hive lock enabled) and the heartbeat (interval iceberg.hive.lock-heartbeat-interval-ms) fails inside persistTable due to HMS connection loss, HMS restart, GC pause longer than lock expiry, or heartbeat thread interruption.

Common situations: Long commits with a short heartbeat interval; flaky network to HMS; embedded/underprovisioned metastore dropping the lock table connection; long pauses (VM/swap) exceeding the lock timeout.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/49371d621ddc2f55. Report an issue: GitHub.

Appendix: source

Thrown at hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java:355

          maxHiveTablePropertySize,
          currentMetadataLocation());

      if (!keepHiveStats) {
        tbl.getParameters().remove(StatsSetupConst.COLUMN_STATS_ACCURATE);
        tbl.getParameters().put(StatsSetupConst.DO_NOT_UPDATE_STATS, StatsSetupConst.TRUE);
      }

      lock.ensureActive();

      try {
        persistTable(
            tbl, updateHiveTable, hiveLockEnabled(base, conf) ? null : baseMetadataLocation);
        lock.ensureActive();

        commitStatus = BaseMetastoreOperations.CommitStatus.SUCCESS;
      } catch (LockException le) {
        commitStatus = BaseMetastoreOperations.CommitStatus.UNKNOWN;
        throw new CommitStateUnknownException(
            "Failed to heartbeat for hive lock while "
                + "committing changes. This can lead to a concurrent commit attempt be able to overwrite this commit. "
                + "Please check the commit history. If you are running into this issue, try reducing "
                + "iceberg.hive.lock-heartbeat-interval-ms.",
            le);
      } catch (org.apache.hadoop.hive.metastore.api.AlreadyExistsException e) {
        throw new AlreadyExistsException(e, "Table already exists: %s.%s", database, tableName);

      } catch (InvalidObjectException e) {
        throw new ValidationException(e, "Invalid Hive object for %s.%s", database, tableName);

      } catch (CommitFailedException | CommitStateUnknownException e) {
        throw e;

      } catch (Throwable e) {
        if (e.getMessage() != null
            && e.getMessage().contains("Table/View 'HIVE_LOCKS' does not exist")) {
          throw new RuntimeException(

View on GitHub (pinned to 86d9c8fc54)