{"record":{"id":"25631b88884a4a5e","repo":"apache/hadoop","slug":"not-able-to-find-the-highest-writable-parent-dir","errorCode":null,"errorMessage":"not able to find the highest writable parent dir","messagePattern":"not able to find the highest writable parent dir","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java","lineNumber":124,"sourceCode":"  public boolean reportChecksumFailure(Path p, FSDataInputStream in,\n                                       long inPos,\n                                       FSDataInputStream sums, long sumsPos) {\n    try {\n      // canonicalize f\n      File f = ((RawLocalFileSystem)fs).pathToFile(p).getCanonicalFile();\n      \n      // find highest writable parent dir of f on the same device\n      String device = new DF(f, getConf()).getMount();\n      File parent = f.getParentFile();\n      File dir = null;\n      while (parent != null && FileUtil.canWrite(parent) &&\n          parent.toString().startsWith(device)) {\n        dir = parent;\n        parent = parent.getParentFile();\n      }\n\n      if (dir==null) {\n        throw new IOException(\n                              \"not able to find the highest writable parent dir\");\n      }\n        \n      // move the file there\n      File badDir = new File(dir, \"bad_files\");\n      if (!badDir.mkdirs()) {\n        if (!badDir.isDirectory()) {\n          throw new IOException(\"Mkdirs failed to create \" + badDir.toString());\n        }\n      }\n      String suffix = \".\" + rand.nextInt();\n      File badFile = new File(badDir, f.getName()+suffix);\n      LOG.warn(\"Moving bad file \" + f + \" to \" + badFile);\n      in.close();                               // close it first\n      boolean b = f.renameTo(badFile);                      // rename it\n      if (!b) {\n        LOG.warn(\"Ignoring failure of renameTo\");\n      }","sourceCodeStart":106,"sourceCodeEnd":142,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalFileSystem.java#L106-L142","documentation":"LocalFileSystem.reportChecksumFailure() runs after a local file fails checksum verification and tries to quarantine the corrupt file under a bad_files directory in the highest writable ancestor that stays on the same mount device (per DF). If no ancestor is both writable and on the device, this IOException is thrown while handling the original corruption.","triggerScenarios":"A checksummed read on LocalFileSystem (file with a .crc sidecar) detects corruption, and every ancestor of the file up to the device root fails FileUtil.canWrite() or leaves the device prefix, so dir stays null.","commonSituations":"Data under root-owned or read-only directories, read-only mounts, hardened containers where the daemon user cannot write anywhere up the tree, unexpected DF mount reporting for the path.","solutions":["Make at least one ancestor directory of the data file writable by the running user (chown/chmod)","Move the data off the read-only mount onto a writable local volume","Inspect `df <path>` and permission bits up the tree, fix, then retry the read (expect the file to still be corrupt — restore from a good copy)"],"exampleFix":"// before: file sits under a read-only dir; quarantine lookup fails\n$ ls -ld /ro-volume/data        # root-owned, no write for daemon user\n// after\n$ sudo chown <procuser> /ro-volume/data && sudo chmod u+w /ro-volume/data","handlingStrategy":"try-catch","validationCode":"File parent = pathToFile(p).getParentFile();\nboolean writableAncestor = false;\nwhile (parent != null) {\n  if (FileUtil.canWrite(parent)) { writableAncestor = true; break; }\n  parent = parent.getParentFile();\n}\nif (!writableAncestor) throw new IOException(\"no writable ancestor for \" + p);","typeGuard":null,"tryCatchPattern":"try {\n  FSDataInputStream in = localFs.open(p);\n} catch (IOException e) {\n  // may carry 'not able to find the highest writable parent dir':\n  // the underlying event is checksum corruption; restore from a clean copy\n}","preventionTips":["Keep local scratch dirs writable by the daemon user","Detect corruption with explicit validation instead of relying on quarantine","Back up local data: quarantine failure leaves the file in place, still corrupt"],"tags":["checksum","local-filesystem","corruption","permissions","quarantine"],"backgroundTag":"no-writable-parent-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}