apache/iceberg · error · RuntimeException

Failed to acquire locks from metastore because the…

Error message

Failed to acquire locks from metastore because the underlying metastore table 'HIVE_LOCKS' does not exist. This can occur when using an embedded metastore which does not support transactions. To fix this use an alternative metastore.

What it means

When using Hive's in-process lock manager, locking data is stored in a transactional 'HIVE_LOCKS' table in the metastore's backing database. If that table does not exist (typical with an embedded metastore, e.g. Derby in non-transactional mode), lock acquisition fails inside doCommit and this RuntimeException is thrown, telling the user to switch metastores.

Solutions

Use a real external metastore (remote HMS service backed by MySQL/Postgres) instead of the embedded one
Disable Hive locking if you don't need it: set iceberg.hive.lock.enabled=false (lockHeartbeat/lock Enabled false path)
Initialize the metastore schema properly (schematool -initSchema) and enable transaction support (hive.txn.manager=DbTxnManager, hive.compactor.initiator.on etc.)
For tests, configure the embedded metastore to support the lock tables or disable locking

Example fix

// before
conf.set("iceberg.hive.lock.enabled", "true"); // with embedded Derby metastore
// after
conf.set("iceberg.hive.lock.enabled", "false");
// or point to a real metastore:
conf.set("hive.metastore.uris", "thrift://metastore-host:9083");

Defensive patterns

Strategy: fallback

Validate before calling

// guard config
boolean embedded = conf.get("hive.metastore.uris", "").isEmpty();
boolean lockEnabled = conf.getBoolean("iceberg.hive.lock.enabled", false);
if (embedded && lockEnabled) throw new IllegalArgumentException("Hive locks require a non-embedded metastore");

Try / catch

try {
  table.newAppend().appendFile(f).commit();
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("HIVE_LOCKS")) {
    conf.set("iceberg.hive.lock.enabled", "false"); // fallback: disable locking
    // or reconfigure to a remote metastore
  } else throw e;
}

Prevention

Never use the embedded metastore with Hive locking enabled in production
Run schematool and enable DbTxnManager when locking is required
Point hive.metastore.uris at a real HMS service
Disable iceberg.hive.lock.enabled if you don't need Hive-side locking

When it happens

Trigger: doCommit with Hive locking enabled (hive.metadata lock enabled / iceberg.hive.lock.enabled) where the lock manager hits 'Table/View HIVE_LOCKS does not exist' — i.e. the HMS backing DB lacks the transaction tables needed by the in-process lock manager.

Common situations: Local/dev setups using the embedded Derby metastore; misconfigured metastore that doesn't run schematool transaction initialization; test environments without a real HMS database.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/d6da3b6aaf89fb4b. Report an issue: GitHub.

Appendix: source

Thrown at hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java:373

        throw new CommitStateUnknownException(
            "Failed to heartbeat for hive lock while "
                + "committing changes. This can lead to a concurrent commit attempt be able to overwrite this commit. "
                + "Please check the commit history. If you are running into this issue, try reducing "
                + "iceberg.hive.lock-heartbeat-interval-ms.",
            le);
      } catch (org.apache.hadoop.hive.metastore.api.AlreadyExistsException e) {
        throw new AlreadyExistsException(e, "Table already exists: %s.%s", database, tableName);

      } catch (InvalidObjectException e) {
        throw new ValidationException(e, "Invalid Hive object for %s.%s", database, tableName);

      } catch (CommitFailedException | CommitStateUnknownException e) {
        throw e;

      } catch (Throwable e) {
        if (e.getMessage() != null
            && e.getMessage().contains("Table/View 'HIVE_LOCKS' does not exist")) {
          throw new RuntimeException(
              "Failed to acquire locks from metastore because the underlying metastore "
                  + "table 'HIVE_LOCKS' does not exist. This can occur when using an embedded metastore which does not "
                  + "support transactions. To fix this use an alternative metastore.",
              e);
        }

        commitStatus = BaseMetastoreOperations.CommitStatus.UNKNOWN;
        if (e.getMessage() != null
            && e.getMessage()
                .contains(
                    "The table has been modified. The parameter value for key '"
                        + HiveTableOperations.METADATA_LOCATION_PROP
                        + "' is")) {
          // It's possible the HMS client incorrectly retries a successful operation, due to network
          // issue for example, and triggers this exception. So we need double-check to make sure
          // this is really a concurrent modification. Hitting this exception means no pending
          // requests, if any, can succeed later, so it's safe to check status in strict mode
          commitStatus = checkCommitStatusStrict(newMetadataLocation, tableMetadata);

View on GitHub (pinned to 86d9c8fc54)