{"record":{"id":"62974e8482afdb15","repo":"apache/iceberg","slug":"failed-to-acquire-lock-on-file-s-with-owner-s","errorCode":null,"errorMessage":"Failed to acquire lock on file: %s with owner: %s","messagePattern":"Failed to acquire lock on file: (.+?) with owner: (.+?)","errorType":"exception","errorClass":"CommitFailedException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java","lineNumber":364,"sourceCode":"      } catch (IOException io) {\n        LOG.warn(\"Error trying to recover the latest version number for {}\", versionHintFile, io);\n        return 0;\n      }\n    }\n  }\n\n  /**\n   * Renames the source file to destination, using the provided file system. If the rename failed,\n   * an attempt will be made to delete the source file.\n   *\n   * @param fs the filesystem used for the rename\n   * @param src the source file\n   * @param dst the destination file\n   */\n  private void renameToFinal(FileSystem fs, Path src, Path dst, int nextVersion) {\n    try {\n      if (!lockManager.acquire(dst.toString(), src.toString())) {\n        throw new CommitFailedException(\n            \"Failed to acquire lock on file: %s with owner: %s\", dst, src);\n      }\n\n      if (fs.exists(dst)) {\n        CommitFailedException cfe =\n            new CommitFailedException(\"Version %d already exists: %s\", nextVersion, dst);\n        RuntimeException re = tryDelete(src);\n        if (re != null) {\n          cfe.addSuppressed(re);\n        }\n\n        throw cfe;\n      }\n\n      if (!fs.rename(src, dst)) {\n        CommitFailedException cfe =\n            new CommitFailedException(\"Failed to commit changes using rename: %s\", dst);\n        RuntimeException re = tryDelete(src);","sourceCodeStart":346,"sourceCodeEnd":382,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java#L346-L382","documentation":"Hadoop commits rename a new metadata file to its final versioned name; a lock manager (default: FileSystemLockManager) serializes this. renameToFinal throws CommitFailedException when acquire() returns false, meaning another writer holds the lock on the destination metadata file, or when the version already exists. This is the expected optimistic-concurrency failure signal for Hadoop tables.","triggerScenarios":"Two writers committing simultaneously to the same Hadoop table; a stale lock left behind by a crashed writer; lock-manager misconfiguration so different writers use inconsistent lock storage; the target version file already existing from a prior commit.","commonSituations":"Concurrent Spark/Flink jobs appending to a table stored on HDFS/local FS; leftover .lock files after a killed job; custom LockManager with a shared store unreachable causing failed acquisition.","solutions":["Retry the commit after a backoff — CommitFailedException means another writer won this round","Remove stale lock files only after confirming no writer is active (check lock owner info)","Configure a shared, consistent LockManager implementation for all writers","Check whether the destination version file already exists; refresh and rebase your commit on the latest metadata"],"exampleFix":"// before\nops.commit(base, metadata); // throws on lock contention\n// after\ntry {\n  ops.commit(base, metadata);\n} catch (CommitFailedException e) {\n  ops.refresh();\n  TableMetadata rebased = applyChanges(ops.current());\n  ops.commit(ops.current(), rebased);\n}","handlingStrategy":"retry","validationCode":"// pre-check lock availability where the LockManager exposes it\nif (!lockManager.tryLock(dst.toString())) { /* back off and retry later */ }","typeGuard":null,"tryCatchPattern":"try { ops.commit(base, metadata); } catch (CommitFailedException e) { backoff(); ops.refresh(); /* rebase and retry */ }","preventionTips":["Use a shared LockManager implementation for all writers","Clean up stale locks only after verifying no active owner","Apply jittered exponential backoff between commit retries","Keep concurrent writers to a Hadoop table limited (or use a real catalog)"],"tags":["locking","concurrency","commit"],"backgroundTag":"lock-acquisition-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}