apache/hadoop · error · HadoopIllegalArgumentException

dfs.datanode.parallel.volumes.load.threads.num = {} < 1

Error message

dfs.datanode.parallel.volumes.load.threads.num = {} < 1

What it means

getParallelVolumeLoadThreadsNum validates dfs.datanode.parallel.volumes.load.threads.num, the thread count used when upgrading/loading data directories in parallel. Any value below 1 is rejected immediately with HadoopIllegalArgumentException naming the key and value; when unset it defaults to the number of data directories.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java:353

    VolumeBuilder builder =
        new VolumeBuilder(this, sd);
    for (NamespaceInfo nsInfo : nsInfos) {
      location.makeBlockPoolDir(nsInfo.getBlockPoolID(), datanode.getConf());

      final BlockPoolSliceStorage bpStorage = getBlockPoolSliceStorage(nsInfo);
      final List<StorageDirectory> dirs = bpStorage.loadBpStorageDirectories(
          nsInfo, location, StartupOption.HOTSWAP, null, datanode.getConf());
      builder.addBpStorageDirectories(nsInfo.getBlockPoolID(), dirs);
    }
    return builder;
  }

  static int getParallelVolumeLoadThreadsNum(int dataDirs, Configuration conf) {
    final String key
        = DFSConfigKeys.DFS_DATANODE_PARALLEL_VOLUME_LOAD_THREADS_NUM_KEY;
    final int n = conf.getInt(key, dataDirs);
    if (n < 1) {
      throw new HadoopIllegalArgumentException(key + " = " + n + " < 1");
    }
    final int min = Math.min(n, dataDirs);
    LOG.info("Using {} threads to upgrade data directories ({}={}, "
        + "dataDirs={})", min, key, n, dataDirs);
    return min;
  }

  static class UpgradeTask {
    private final StorageLocation dataDir;
    private final Future<StorageDirectory> future;

    UpgradeTask(StorageLocation dataDir, Future<StorageDirectory> future) {
      this.dataDir = dataDir;
      this.future = future;
    }
  }

  /**

View on GitHub (pinned to 2add963021)

Solutions

1. Set the property to a positive integer (1 for serial loading, roughly the disk count for parallel)
2. Or remove the property entirely to get the default (number of data dirs)
3. Lint hdfs-site.xml in CI so numeric floors are checked before deploy

Example fix

<!-- before -->
<property>
  <name>dfs.datanode.parallel.volumes.load.threads.num</name>
  <value>0</value>
</property>
<!-- after -->
<property>
  <name>dfs.datanode.parallel.volumes.load.threads.num</name>
  <value>1</value>
</property>

int n = conf.getInt(
    DFSConfigKeys.DFS_DATANODE_PARALLEL_VOLUME_LOAD_THREADS_NUM_KEY, numDataDirs);
if (n < 1) {
  throw new IllegalArgumentException(
      DFSConfigKeys.DFS_DATANODE_PARALLEL_VOLUME_LOAD_THREADS_NUM_KEY + " must be >= 1, got " + n);
}

catch (HadoopIllegalArgumentException e) {
  // config error: fix hdfs-site.xml value, no retry will help
  failStartupWithConfigGuidance(e);
}

When it happens

Trigger: Setting dfs.datanode.parallel.volumes.load.threads.num to 0 or a negative number in hdfs-site.xml, then starting the datanode or entering any code path that computes parallel volume-load threads (e.g. upgrade via recoverTransitionRead).

Common situations: Copy-paste config with a stray 0; trying to 'disable parallelism' by setting 0 (not supported - use 1); templated config where an empty variable parses as 0.

Related errors

• Not a valid Boolean value for {property} in reconfSlowPeerPa
• Not a valid Boolean value for {property}
• No directory is specified.
• Invalid value configured for dfs.datanode.failed.volumes.tol
• Security is enabled but block access tokens (via dfs.block.a

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/08e581f0c26c741b. Report an issue: GitHub.