apache/iceberg · error · LockException

Could not acquire the lock on %s.%s, lock request ended in s

Error message

Could not acquire the lock on %s.%s, lock request ended in state %s

What it means

MetastoreLock acquires an exclusive Hive metastore lock for a table commit. After issuing the lock request, the code polls until the lock is acquired; if the loop exits without an acquired lock and without a recorded thrift error, this safety-net LockException is thrown reporting the final lock state. It indicates the lock request ended in an unexpected state (e.g. WAITING/ABORT) rather than ACQUIRED.

Source

Thrown at hive-metastore/src/main/java/org/apache/iceberg/hive/MetastoreLock.java:254

    } finally {
      if (!lockInfo.lockState.equals(LockState.ACQUIRED)) {
        unlock(Optional.of(lockInfo.lockId));
      }
    }

    if (!lockInfo.lockState.equals(LockState.ACQUIRED)) {
      if (timeout) {
        throw new LockException(
            "Timed out after %s ms waiting for lock on %s.%s", duration, databaseName, tableName);
      }

      if (thriftError != null) {
        throw new LockException(
            thriftError, "Metastore operation failed for %s.%s", databaseName, tableName);
      }

      // Just for safety. We should not get here.
      throw new LockException(
          "Could not acquire the lock on %s.%s, lock request ended in state %s",
          databaseName, tableName, lockInfo.lockState);
    } else {
      return lockInfo.lockId;
    }
  }

  /**
   * Creates a lock, retrying if possible on failure.
   *
   * @return The {@link LockInfo} object for the successfully created lock
   * @throws LockException When we are not able to fill the hostname for lock creation, or there is
   *     an error during lock creation
   */
  @SuppressWarnings("ReverseDnsLookup")
  private LockInfo createLock() throws LockException {
    LockInfo lockInfo = new LockInfo();

View on GitHub (pinned to 86d9c8fc54)

Solutions

Check for concurrent writers/engines holding locks on the table and reduce contention (SHOW LOCKS on the table in Hive).
Increase the lock acquisition retry/wait budget and Hive lock timeout so transient WAITING states resolve to ACQUIRED.
Inspect lockInfo.lockState in the message: if ABORTED, the request was rejected — re-run the commit; if still WAITING, tune metastore lock checker intervals.
Verify Hive metastore connectivity and health; persistent failures usually surface as thriftError instead, but flaky RPCs can leave odd states.

Example fix

// before
Table table = HiveCatalog.loadTable(identifier);
table.refresh(); // retry commit manually after lock failure
// after
Tasks.foreach(table).retry(3).exponentialBackoff(1000, 15000)
    .throwFailureWhenFinished();
RefreshOperation... table.refresh(); table.commit(); // re-attempt under lock

Defensive patterns

Strategy: retry

Validate before calling

// check for competing locks before committing
// in Hive: SHOW LOCKS <db>.<table>;  or via IMetaStoreClient showLocks()
// proceed only if no EXCLUSIVE/WAITING locks exist

Try / catch

try {
  table.refresh();
  table.commit(apply);
} catch (LockException | CommitFailedException e) {
  // inspect lockState in message; backoff and retry
  backoffRetry(() -> { table.refresh(); table.commit(apply); });
}

Prevention

Minimize concurrent writers to the same Hive-catalog table
Keep commits short-lived; avoid slow scan planning inside the lock window
Tune hive.txn.timeout and lock retry budgets above worst-case commit duration
Monitor SHOW LOCKS output for stuck WAITING/ABORT states

When it happens

Trigger: Calling commit on a Hive-catalog table when the metastore lock request terminates in a non-ACQUIRED state: the lock is aborted by another holder, the wait loop exhausts, or Hive returns a state the polling loop does not handle.

Common situations: Concurrent writers contending for the same table until one lock is aborted; Hive metastore under heavy load returning slow lock acquisition; long-running commits exceeding Hive's lock timeout (hive.txn.timeout) so the lock is reaped while waiting.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/9c8b80b93c1a63c7. Report an issue: GitHub.