{"record":{"id":"a77e11be03a9f370","repo":"apache/hadoop","slug":"src-tostring-no-such-file-or-directory","errorCode":null,"errorMessage":"src.toString() + \": No such file or directory\"","messagePattern":"src\\.toString\\(\\) \\+ \": No such file or directory\"","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java","lineNumber":549,"sourceCode":"      }\n    } else if (src.isFile()) {\n      InputStream in = null;\n      OutputStream out =null;\n      try {\n        in = Files.newInputStream(src.toPath());\n        out = dstFS.create(dst);\n        IOUtils.copyBytes(in, out, conf);\n      } catch (IOException e) {\n        IOUtils.closeStream( out );\n        IOUtils.closeStream( in );\n        throw e;\n      }\n    } else if (!src.canRead()) {\n      throw new IOException(src.toString() +\n                            \": Permission denied\");\n\n    } else {\n      throw new IOException(src.toString() +\n                            \": No such file or directory\");\n    }\n    if (deleteSource) {\n      return FileUtil.fullyDelete(src);\n    } else {\n      return true;\n    }\n  }\n\n  /**\n   * Copy FileSystem files to local files.\n   *\n   * @param srcFS srcFs.\n   * @param src src.\n   * @param dst dst.\n   * @param deleteSource delete source.\n   * @param conf configuration.\n   * @throws IOException raised on errors performing I/O.","sourceCodeStart":531,"sourceCodeEnd":567,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java#L531-L567","documentation":"Thrown by FileUtil.copy(File src, FileSystem dstFS, Path dst, ...) when the local source path is neither a directory (isDirectory()) nor a regular file (isFile()) but File.canRead() still returns true. The fall-through else branch mimics the shell error '<path>: No such file or directory'. In practice it fires for special files (FIFOs, sockets, device nodes like /dev/null) or a source deleted/raced between the isFile() check and canRead() check. Note the sibling branch: a truly missing file usually fails canRead() first and reports 'Permission denied' instead, so this specific message is the rarer race/special-file case.","triggerScenarios":"Calling FileUtil.copy(localFile, remoteFS, dstPath, deleteSource, conf) where localFile is a character device, FIFO, or socket; recursive directory copy where a listed file is unlinked between listFiles() and copy(); passing a symlink whose target is swapped mid-copy.","commonSituations":"Scripts copying a directory tree that another process (log rotation, tmp cleaner) is concurrently deleting; passing /dev/null or a named pipe as source; TOCTOU races in test harnesses that stage and copy files quickly.","solutions":["Pre-check the source before copying: require src.exists() && (src.isFile() || src.isDirectory()) and fail fast with your own message","For directories being copied while mutated, snapshot or lock the tree first (copy from a stable location) to avoid the isFile/canRead race","Exclude or special-case non-regular files (FIFOs, devices) instead of passing them to FileUtil.copy","If the source should exist, verify the path spelling/getCanonicalFile() and re-run; if it intermittently vanishes, add a bounded retry around the whole copy"],"exampleFix":"// before\nboolean ok = FileUtil.copy(srcFile, fs, dstPath, false, conf);\n\n// after\nif (!srcFile.exists() || !(srcFile.isFile() || srcFile.isDirectory())) {\n  throw new FileNotFoundException(srcFile + \": not a copyable regular file/dir\");\n}\nboolean ok = FileUtil.copy(srcFile, fs, dstPath, false, conf);","handlingStrategy":"validation","validationCode":"private static void requireCopyableSource(File src) throws IOException {\n  if (!src.exists()) throw new FileNotFoundException(src.toString());\n  if (!src.isFile() && !src.isDirectory()) {\n    throw new IOException(src + \" is not a regular file or directory\");\n  }\n}\n// call before FileUtil.copy(src, dstFS, dst, deleteSource, conf)","typeGuard":"static boolean isCopyableLocalSource(File f) {\n  return f.isFile() || f.isDirectory(); // excludes FIFOs, devices, sockets\n}","tryCatchPattern":"try {\n  FileUtil.copy(src, fs, dst, false, conf);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().endsWith(\"No such file or directory\")) {\n    // source vanished or is a special file; re-stage and retry once\n  } else throw e;\n}","preventionTips":["Never pass device nodes, FIFOs, or sockets to FileUtil.copy; filter with isFile()","Stage the tree to copy into a stable directory no other process mutates","Remember the missing-file case often reports 'Permission denied' from this method — check File.exists() yourself for accurate messages"],"tags":["local-file","copy","file-status-race","hadoop-common","fileutil"],"backgroundTag":"file-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}