{"record":{"id":"8141a6806847d00c","repo":"apache/hadoop","slug":"unable-to-download-to-any-storage-directory","errorCode":null,"errorMessage":"Unable to download to any storage directory","messagePattern":"Unable to download to any storage directory","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java","lineNumber":260,"sourceCode":"              LOG.warn(\"Overwriting existing file \" + f\n                  + \" with file downloaded from \" + url);\n            }\n            FileOutputStream fos = new FileOutputStream(f);\n            outputStreams.add(fos);\n            streamPathMap.put(fos, f);\n          } catch (IOException ioe) {\n            LOG.warn(\"Unable to download file \" + f, ioe);\n            // This will be null if we're downloading the fsimage to a file\n            // outside of an NNStorage directory.\n            if (dstStorage != null &&\n                (dstStorage instanceof StorageErrorReporter)) {\n              ((StorageErrorReporter)dstStorage).reportErrorOnFile(f);\n            }\n          }\n        }\n\n        if (outputStreams.isEmpty()) {\n          throw new IOException(\n              \"Unable to download to any storage directory\");\n        }\n      }\n\n      byte[] buf = new byte[IO_FILE_BUFFER_SIZE];\n      while (num > 0) {\n        num = stream.read(buf);\n        if (num > 0) {\n          received += num;\n          for (FileOutputStream fos : outputStreams) {\n            fos.write(buf, 0, num);\n          }\n          if (throttler != null) {\n            throttler.throttle(num);\n          }\n        }\n      }\n      finishedReceiving = true;","sourceCodeStart":242,"sourceCodeEnd":278,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java#L242-L278","documentation":"Before streaming, Util.receiveFile opens a FileOutputStream on every candidate storage path; each failure is logged as WARN 'Unable to download file <f>' and the directory is reported to StorageErrorReporter. If not a single stream could be opened, the download cannot proceed and this IOException is thrown. The real root cause is always in the preceding WARN stack traces, not in this message.","triggerScenarios":"Every target path fails FileOutputStream creation: permission denied on the storage dir, parent directory missing, path occupied by a directory, disk full, or read-only filesystem.","commonSituations":"NameNode/2NN restarted as a different user so dfs.namenode.name.dir entries are no longer writable; storage volume full or read-only after an fs error; storage directory deleted but not recreated; SELinux denial on the name dirs.","solutions":["Read the WARN 'Unable to download file …' lines immediately before this exception — they carry the per-path cause (Permission denied, ENOSPC, ENOENT…)","Fix ownership/permissions: chown -R hdfs:hdfs the storage dirs and chmod so the NN user can write","Free disk space or remount the volume read-write; recreate missing parent directories","Verify the path is a directory and not a regular file occupying the name"],"exampleFix":"# before: storage dir owned by root\nls -ld /data/nn/current   # drwxr-xr-x root root\n\n# after\nchown -R hdfs:hdfs /data/nn\nchmod 755 /data/nn","handlingStrategy":"validation","validationCode":"// Verify every destination is writable before starting the transfer\nfor (File dir : storageDirs) {\n  Path p = dir.toPath();\n  if (!Files.isDirectory(p) || !Files.isWritable(p)) {\n    throw new IllegalStateException(\"Storage dir not writable: \" + p);\n  }\n  if (p.getFileSystem().getUsableSpace(p) < minFreeBytes) {\n    throw new IllegalStateException(\"Storage dir low on space: \" + p);\n  }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Monitor free space and ownership of all dfs.namenode.name.dir volumes","Run the NameNode/checkpointer under a fixed service account that owns the storage dirs","Add a pre-start readiness check that touches a probe file in each storage dir"],"tags":["hdfs","storage-directory","permissions","disk-full","checkpoint","fsimage"],"backgroundTag":"storage-directory-unwritable","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}