apache/hadoop · error · HadoopIllegalArgumentException

Invalid value configured for dfs.datanode.disk.check.timeout

Error message

Invalid value configured for dfs.datanode.disk.check.timeout - {} (should be >= 0)

What it means

DatasetVolumeChecker reads dfs.datanode.disk.check.timeout a second time into diskCheckTimeout (used as the per-check timeout handed to ThrottledAsyncChecker) and validates it with >= 0. Because the earlier validation of the same key (line 123) already throws for values <= 0, this specific >= 0 branch is effectively dead code: any value that could trigger it (negative) would have aborted startup earlier with the 'should be > 0' message.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/checker/DatasetVolumeChecker.java:152

    minDiskCheckGapMs = conf.getTimeDuration(
        DFSConfigKeys.DFS_DATANODE_DISK_CHECK_MIN_GAP_KEY,
        DFSConfigKeys.DFS_DATANODE_DISK_CHECK_MIN_GAP_DEFAULT,
        TimeUnit.MILLISECONDS);

    if (minDiskCheckGapMs < 0) {
      throw new HadoopIllegalArgumentException("Invalid value configured for "
          + DFS_DATANODE_DISK_CHECK_MIN_GAP_KEY + " - "
          + minDiskCheckGapMs + " (should be >= 0)");
    }

    diskCheckTimeout = conf.getTimeDuration(
        DFSConfigKeys.DFS_DATANODE_DISK_CHECK_TIMEOUT_KEY,
        DFSConfigKeys.DFS_DATANODE_DISK_CHECK_TIMEOUT_DEFAULT,
        TimeUnit.MILLISECONDS);

    if (diskCheckTimeout < 0) {
      throw new HadoopIllegalArgumentException("Invalid value configured for "
          + DFS_DATANODE_DISK_CHECK_TIMEOUT_KEY + " - "
          + diskCheckTimeout + " (should be >= 0)");
    }

    lastAllVolumesCheck = timer.monotonicNow() - minDiskCheckGapMs;

    if (maxVolumeFailuresTolerated < DataNode.MAX_VOLUME_FAILURE_TOLERATED_LIMIT) {
      throw new HadoopIllegalArgumentException("Invalid value configured for "
          + DFS_DATANODE_FAILED_VOLUMES_TOLERATED_KEY + " - "
          + maxVolumeFailuresTolerated + " "
          + DataNode.MAX_VOLUME_FAILURES_TOLERATED_MSG);
    }

    delegateChecker = new ThrottledAsyncChecker<>(
        timer, minDiskCheckGapMs, diskCheckTimeout,
        Executors.newCachedThreadPool(
            new ThreadFactoryBuilder()
                .setNameFormat("DataNode DiskChecker thread %d")

View on GitHub (pinned to 2add963021)

Solutions

1. If you maintain a fork, unify the duplicate reads of dfs.datanode.disk.check.timeout into one field with one validation (> 0)
2. If hit as a user: fix dfs.datanode.disk.check.timeout to a positive duration anyway - it satisfies both checks
3. Compare your deployed hadoop-hdfs jar against the upstream version to detect divergent/patched code

long t = conf.getTimeDuration(
    DFSConfigKeys.DFS_DATANODE_DISK_CHECK_TIMEOUT_KEY,
    DFSConfigKeys.DFS_DATANODE_DISK_CHECK_TIMEOUT_DEFAULT, TimeUnit.MILLISECONDS);
if (t <= 0) throw new IllegalStateException(
    "dfs.datanode.disk.check.timeout must be > 0, got " + t); // satisfies both validations

When it happens

Trigger: Practically unreachable in a normal build: it would require dfs.datanode.disk.check.timeout to be negative while the identical earlier read was non-negative (e.g., a custom Configuration whose value changes between getTimeDuration calls, or a patched/branch-divergent DatasetVolumeChecker where the first check reads a different key).

Common situations: Custom Hadoop forks or patches where HDFS-14123-era refactoring changed one of the two reads to a different key, resurrecting this branch; static analysis or error-catalog tooling surfacing the message string. If you see this exact message, you are likely running modified code.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Invalid value configured for dfs.datanode.disk.check.timeout
• Invalid value configured for dfs.datanode.disk.check.min.gap
• Decay Factor must be between 0 and 1
• Period millis must be >= 0
• Cannot lock storage {root}. The directory is already locked

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7386c4f3998563ce. Report an issue: GitHub.