{"record":{"id":"76aea56353b14d8d","repo":"apache/hadoop","slug":"file-already-exists-s","errorCode":null,"errorMessage":"File already exists: %s","messagePattern":"File already exists: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java","lineNumber":562,"sourceCode":"   */\n  @Override\n  public FSDataOutputStream create(Path f, FsPermission permission,\n      boolean overwrite, int bufferSize, short replication, long blockSize,\n      Progressable progress) throws IOException {\n    final ChannelSftp client = connect();\n    Path workDir;\n    try {\n      workDir = new Path(client.pwd());\n    } catch (SftpException e) {\n      throw new IOException(e);\n    }\n    Path absolute = makeAbsolute(workDir, f);\n    if (exists(client, f)) {\n      if (overwrite) {\n        delete(client, f, false);\n      } else {\n        disconnect(client);\n        throw new IOException(String.format(E_FILE_EXIST, f));\n      }\n    }\n    Path parent = absolute.getParent();\n    if (parent == null || !mkdirs(client, parent, FsPermission.getDefault())) {\n      parent = (parent == null) ? new Path(\"/\") : parent;\n      disconnect(client);\n      throw new IOException(String.format(E_CREATE_DIR, parent));\n    }\n    OutputStream os;\n    try {\n      final String previousCwd = client.pwd();\n      client.cd(parent.toUri().getPath());\n      os = client.put(f.getName());\n      client.cd(previousCwd);\n    } catch (SftpException e) {\n      throw new IOException(e);\n    }\n    FSDataOutputStream fos = new FSDataOutputStream(os, statistics) {","sourceCodeStart":544,"sourceCodeEnd":580,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java#L544-L580","documentation":"Thrown by SFTPFileSystem.create when the target file already exists on the SFTP server and the overwrite parameter is false. Note the adapter throws a raw IOException rather than FileAlreadyExistsException, so callers that catch the specialized type will miss it.","triggerScenarios":"Calling fs.create(f, bufferSize, replication, blockSize, progress) or any overload with overwrite=false when exists(client, f) is true on the sftp:// server; i.e. writing an output path that a previous run already produced.","commonSituations":"Re-running a failed job without cleaning its output directory; frameworks whose committers rely on FileAlreadyExistsException detection; two writers racing to the same target path.","solutions":["Pass overwrite=true when the write should replace prior content: fs.create(f, true).","Delete the existing file first (fs.delete(f, false)) when you need a deliberate clobber point.","If you must detect this case programmatically, catch IOException and test the message for 'File already exists' — the type is not FileAlreadyExistsException in this adapter."],"exampleFix":"// before\nFSDataOutputStream out = fs.create(outPath, false, progress); // throws on rerun\n\n// after\nFSDataOutputStream out = fs.create(outPath, true, progress);","handlingStrategy":"validation","validationCode":"if (fs.exists(outPath) && !overwrite) {\n  throw new IOException(\"refusing to overwrite existing \" + outPath);\n}\nFSDataOutputStream out = fs.create(outPath, overwrite);","typeGuard":null,"tryCatchPattern":"try {\n  out = fs.create(outPath, false);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"File already exists\")) {\n    out = fs.create(outPath, true); // or surface 'output exists' to the user\n  } else {\n    throw e;\n  }\n}","preventionTips":["Delete or archive prior output before reruns instead of relying on create(false).","Remember this adapter throws plain IOException, not FileAlreadyExistsException — match on message."],"tags":["sftp","create","file-exists","overwrite","hadoop"],"backgroundTag":"file-already-exists","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}