apache/hadoop · error · IOException

delay={} is negative. Please check {}

Error message

delay={} is negative. Please check {}

What it means

checkConfiguration reads the fed-rename trash delay from dfs.federation.router.federation.rename.delay; the default (1000, in ms) is positive, so this IOException only fires when an operator explicitly sets the key to a negative number. The delay controls how long rename waits for disallowed/open-file conditions before force handling in the distcp procedure.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterFederationRename.java:280

    routerRenameCounter.decrementAndGet();
  }

  static void checkConfiguration(Configuration conf) throws IOException {
    int map = conf.getInt(DFS_ROUTER_FEDERATION_RENAME_MAP, -1);
    int bandwidth = conf.getInt(DFS_ROUTER_FEDERATION_RENAME_BANDWIDTH, -1);
    long delay = conf.getLong(DFS_ROUTER_FEDERATION_RENAME_DELAY,
        DFS_ROUTER_FEDERATION_RENAME_DELAY_DEFAULT);
    int diff = conf.getInt(DFS_ROUTER_FEDERATION_RENAME_DIFF,
        DFS_ROUTER_FEDERATION_RENAME_DIFF_DEFAULT);
    if (map < 0) {
      throw new IOException("map=" + map + " is negative. Please check "
          + DFS_ROUTER_FEDERATION_RENAME_MAP);
    } else if (bandwidth < 0) {
      throw new IOException(
          "bandwidth=" + bandwidth + " is negative. Please check "
              + DFS_ROUTER_FEDERATION_RENAME_BANDWIDTH);
    } else if (delay < 0) {
      throw new IOException("delay=" + delay + " is negative. Please check "
          + DFS_ROUTER_FEDERATION_RENAME_DELAY);
    } else if (diff < 0) {
      throw new IOException("diff=" + diff + " is negative. Please check "
          + DFS_ROUTER_FEDERATION_RENAME_DIFF);
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

  1. Set dfs.federation.router.federation.rename.delay to a non-negative value (default 1000 ms) or remove the key to fall back to the default
  2. Audit the router's hdfs-site.xml for other fed-rename keys set to -1 (map, bandwidth, diff) and fix them in the same pass
  3. Restart the router after correcting the configuration

Example fix

# before
<property><name>dfs.federation.router.federation.rename.delay</name><value>-1</value></property>
# after
<property><name>dfs.federation.router.federation.rename.delay</name><value>1000</value></property>  <!-- or omit the key entirely -->
Defensive patterns

Strategy: validation

Validate before calling

long delay = conf.getLong("dfs.federation.router.federation.rename.delay", 1000L);
if (delay < 0) {
  throw new IllegalStateException(
      "dfs.federation.router.federation.rename.delay must be non-negative (default 1000 ms)");
}

Try / catch

try {
  routerFedRename.routerFedRename(src, dst, srcLocs, dstLocs);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("dfs.federation.router.federation.rename.delay")) {
    // remove the negative override or set it to the default, restart router
  }
  throw e;
}

Prevention

When it happens

Trigger: dfs.federation.router.federation.rename.delay is explicitly configured to a negative long on the router and a federation rename triggers checkConfiguration.

Common situations: Copy-pasted example configs with placeholder negative values; units confusion leading someone to write -1 meaning 'no delay'; global config templating that injects -1 defaults for optional numeric keys.

Related errors


AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/40ca12c0f6984597. Report an issue: GitHub.