apache/hadoop · error · DiskErrorException

Could not find any valid local directory for ${pathStr} with

Error message

Could not find any valid local directory for ${pathStr} with requested size ${size} on host ${hostname} as the max capacity in any directory (${pathNames} ) is ${maxCapacity}

What it means

Error "Could not find any valid local directory for ${pathStr} with requested size ${size} on host ${hostname} as the max capacity in any directory (${pathNames} ) is ${maxCapacity}" thrown in apache/hadoop.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java:572

      }
      if (returnPath != null) {
        return returnPath;
      }
      
      //no path found
      String hostname = getHostname();
      String newErrorText = "Could not find any valid local directory for "
          + pathStr + " with requested size " + size
          + " on host " + hostname
          + " as the max capacity in any directory"
          + " (" + pathNames + " )"
          + " is " + maxCapacity;
      if (errorText != null) {
        newErrorText = newErrorText + " due to " + errorText;
      }
      LOG.error(newErrorText);
      LOG.error(history.toString());
      throw new DiskErrorException(newErrorText, diskException);
    }

    /** Creates a file on the local FS. Pass size as 
     * {@link LocalDirAllocator.SIZE_UNKNOWN} if not known apriori. We
     *  round-robin over the set of disks (via the configured dirs) and return
     *  a file on the first path which has enough space. The file is guaranteed
     *  to go away when the JVM exits.
     */
    public File createTmpFileForWrite(String pathStr, long size, 
        Configuration conf) throws IOException {

      // find an appropriate directory
      Path path = getLocalPathForWrite(pathStr, size, conf, true);
      File dir = new File(path.getParent().toUri().getPath());
      String prefix = path.getName();

      // create a temp file on this directory
      File result = File.createTempFile(prefix, null, dir);

View on GitHub (pinned to 2add963021)

Solutions

1. Request a smaller size or free space in the configured local directories.
2. Add larger-capacity directories to the local-dir configuration.
3. Review max capacity reporting; a nearly full disk triggers this even for modest requests.

When it happens

Trigger: Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java:572 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c70b2d957ef4ba07. Report an issue: GitHub.