What causes "Failed to release lock on file: {} with owner: {}" in apache/iceberg?

Inspect the lock store for a stale lock on the destination metadata file and remove it manually

Failed to release lock on file: {} with owner: {}

Error message

Failed to release lock on file: {} with owner: {}

What it means

During renameToFinal(), HadoopTableOperations commits a new metadata file by locking the destination, writing, then renaming. If the lock is not successfully released in the finally block (lockManager.release returns false), this warning is logged. The commit itself already succeeded or failed; the risk is a stale lock that can block future commits until it expires.

Source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java:398

        CommitFailedException cfe =
            new CommitFailedException("Failed to commit changes using rename: %s", dst);
        RuntimeException re = tryDelete(src);
        if (re != null) {
          cfe.addSuppressed(re);
        }
        throw cfe;
      }
    } catch (IOException e) {
      CommitFailedException cfe =
          new CommitFailedException(e, "Failed to commit changes using rename: %s", dst);
      RuntimeException re = tryDelete(src);
      if (re != null) {
        cfe.addSuppressed(re);
      }
      throw cfe;
    } finally {
      if (!lockManager.release(dst.toString(), src.toString())) {
        LOG.warn("Failed to release lock on file: {} with owner: {}", dst, src);
      }
    }
  }

  /**
   * Deletes the file from the file system. Any RuntimeException will be caught and returned.
   *
   * @param path the file to be deleted.
   * @return RuntimeException caught, if any. null otherwise.
   */
  private RuntimeException tryDelete(Path path) {
    try {
      io().deleteFile(path.toString());
      return null;
    } catch (RuntimeException re) {
      return re;
    }
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

Inspect the lock store for a stale lock on the destination metadata file and remove it manually
Verify all writers use identical lock manager configuration (e.g. same lock table settings)
Reduce commit duration or increase lock TTL/expiry
Ensure no two jobs write to the same table concurrently with different owner settings
Retry the commit once the lock has expired

Example fix

// before (mismatched owners across jobs)
conf.set("lock.owner", "job-a")  // other jobs use job-b owner
// after
conf.set("iceberg.lock.owner", CLUSTER_WIDE_OWNER_ID)  // consistent for all writers

Defensive patterns

Strategy: retry

Validate before calling

// before committing, ensure no stale lock exists on the metadata path
if (lockStore.hasActiveLock(dstPath)) {
  throw new IllegalStateException("stale lock on " + dstPath);
}

Try / catch

try {
  table.fastAppend().commit();
} catch (CommitFailedException e) {
  // a stale lock blocked the commit; clear lock store and retry with backoff
}

Prevention

Use identical lock manager config across all writer jobs
Set lock TTL comfortably above max commit duration
Periodically garbage-collect stale locks from the lock store
Avoid running two independent jobs writing the same table without catalog-level locking

When it happens

Trigger: commit() -> renameToFinal() where lockManager.release(dst, src) returns false — e.g. the lock owner token no longer matches, lock state was corrupted, or the underlying lock store operation silently failed while returning false.

Common situations: Long-running commits that exceeded lock TTL so ownership lapsed; duplicated writer processes using mismatched lock owner configuration; stale locks left by crashed previous runs interfering with release bookkeeping.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/e708db5f1fe71a1c. Report an issue: GitHub.