apache/hadoop · error · IllegalArgumentException

Failed to read {type} node list from file: {filename}

Error message

Failed to read {type} node list from file: {filename}

What it means

Thrown when HostsFileReader.readFileToSet(type, filename, nodes) raises IOException while loading a node-list file given via '-f <filename>' after one of the node-filter options (-include/-exclude/-source/-excludeSource/-target/-excludeTarget). The IOException is wrapped in IllegalArgumentException naming the list type and file path. readFileToSet opens a plain local file, so the file must exist and be readable on the gateway machine running the balancer - not in HDFS.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java:1234

      }
      return b.build();
    }

    private static int processHostList(String[] args, int i, String type,
        Set<String> nodes) {
      Preconditions.checkArgument(++i < args.length,
          "List of %s nodes | -f <filename> is missing: args=%s",
          type, Arrays.toString(args));
      if ("-f".equalsIgnoreCase(args[i])) {
        Preconditions.checkArgument(++i < args.length,
            "File containing %s nodes is not specified: args=%s",
            type, Arrays.toString(args));

        final String filename = args[i];
        try {
          HostsFileReader.readFileToSet(type, filename, nodes);
        } catch (IOException e) {
          throw new IllegalArgumentException(
              "Failed to read " + type + " node list from file: " + filename);
        }
      } else {
        final String[] addresses = StringUtils.getTrimmedStrings(args[i]);
        nodes.addAll(Arrays.asList(addresses));
      }
      return i;
    }

    private static Set<String> parseBlockPoolList(String string) {
      String[] addrs = StringUtils.getTrimmedStrings(string);
      return new HashSet<String>(Arrays.asList(addrs));
    }

    private static void printUsage(PrintStream out) {
      out.println(USAGE + "\n");
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the file exists and is readable by the balancer user on the exact machine where the balancer runs (use an absolute path)
2. Use one host (optionally host:port or host:storageType per line for include/exclude lists) per line
3. If the list lives in HDFS, copy it to local disk first, e.g. 'hdfs dfs -get /ops/balancer-include.txt /tmp/inc.txt', then pass the local path

Example fix

# before
hdfs balancer -include -f /ops/include.txt   # HDFS path, fails

# after
hdfs dfs -get /ops/include.txt /tmp/include.txt
hdfs balancer -include -f /tmp/include.txt

static Path validateNodeListFile(String filename) {
  Path p = Paths.get(filename);
  if (!Files.isRegularFile(p) || !Files.isReadable(p)) {
    throw new IllegalArgumentException("Node list file missing/unreadable (must be local): " + p);
  }
  return p;
}

catch (IllegalArgumentException e) { log.error("Could not load node list: {}", e.getMessage()); System.exit(-1); }

When it happens

Trigger: Passing e.g. 'hdfs balancer -include -f /tmp/inc.txt' where the file is missing, unreadable (permissions), points to an HDFS path, or the path has a typo. Any open/read IOException from the local filesystem triggers it.

Common situations: Operators assuming '-f' reads from HDFS; files staged on a different gateway than where the balancer job runs; permission mismatches when the balancer runs as a different user; cron environments with a different working directory and relative paths.

Related errors

• Number out of range: threshold = {threshold}
• args = {args}
• Cannot parse string "{s}"
• Illegal option {}
• Not enough arguments: expected {} but got {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/481be514a158b2e3. Report an issue: GitHub.