apache/hadoop · error · FileConflictException

File conflicts during rename

Error message

File conflicts during rename, {}

What it means

An ObsException with response code CONFLICT from the POSIX RenameRequest becomes FileConflictException('File conflicts during rename, <responseStatus>'). POSIX-protocol OBS buckets perform a server-side atomic rename that never overwrites: if the destination key already exists, the service rejects the rename.

Solutions

1. Check fs.exists(dst) first and delete the destination when overwrite is intended
2. Make commits idempotent: missing source plus existing destination means the commit already happened
3. Use unique per-attempt destination names and finalize with a rename of a not-yet-existing key

Example fix

// before
fs.rename(src, dst);

// after
if (fs.exists(dst)) {
  fs.delete(dst, false);
}
fs.rename(src, dst);

if (fs.exists(dst)) {
  fs.delete(dst, false); // deliberate overwrite of an existing destination
}

try {
  fs.rename(src, dst);
} catch (FileConflictException e) {
  if (fs.exists(dst)) {
    // destination taken: delete it or pick a new name, then retry once
  } else {
    throw e;
  }
}

When it happens

Trigger: rename(src, dst) where dst already exists; two clients renaming different sources onto the same destination concurrently; a retried commit whose destination was already written by the first attempt.

Common situations: Commit-by-rename algorithms ported from filesystems with overwrite semantics; concurrent writers to one output path; exactly-once sink implementations that assume rename is idempotent.

Related errors

• is not a directory
• No such file or directory
• destination has no parent
• destination is an existed file
• destination parent [ ] is not a directory

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/33da429fc20913cf. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSPosixBucketUtils.java:282

  static void innerFsRenameFile(final OBSFileSystem owner,
      final String srcKey,
      final String dstKey) throws IOException {
    LOG.debug("RenameFile path {} to {}", srcKey, dstKey);

    try {
      final RenameRequest renameObjectRequest = new RenameRequest();
      renameObjectRequest.setBucketName(owner.getBucket());
      renameObjectRequest.setObjectKey(srcKey);
      renameObjectRequest.setNewObjectKey(dstKey);
      owner.getObsClient().renameFile(renameObjectRequest);
      owner.getSchemeStatistics().incrementWriteOps(1);
    } catch (ObsException e) {
      if (e.getResponseCode() == OBSCommonUtils.NOT_FOUND_CODE) {
        throw new FileNotFoundException(
            "No such file or directory: " + srcKey);
      }
      if (e.getResponseCode() == OBSCommonUtils.CONFLICT_CODE) {
        throw new FileConflictException(
            "File conflicts during rename, " + e.getResponseStatus());
      }
      throw OBSCommonUtils.translateException(
          "renameFile(" + srcKey + ", " + dstKey + ")", srcKey, e);
    }
  }

  /**
   * Used to rename a source object to a destination object which is not existed
   * before rename.
   *
   * @param owner  OBS File System instance
   * @param srcKey source object key
   * @param dstKey destination object key
   * @throws IOException io exception
   */
  static void fsRenameToNewObject(final OBSFileSystem owner,
      final String srcKey,

View on GitHub (pinned to 2add963021)