apache/hadoop · error · StandbyException

Router {} is overloaded for NS: {}

Error message

Router {} is overloaded for NS: {}

What it means

Error "Router {} is overloaded for NS: {}" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java:1871

   * @throws IOException If permit could not be acquired for the nsId.
   */
  protected void acquirePermit(final String nsId, final UserGroupInformation ugi,
      final String methodName, RouterRpcFairnessPolicyController controller)
      throws IOException {
    if (controller != null) {
      if (!controller.acquirePermit(nsId)) {
        // Throw StandByException,
        // Clients could fail over and try another router.
        if (rpcMonitor != null) {
          rpcMonitor.proxyOpPermitRejected(nsId);
        }
        incrRejectedPermitForNs(nsId);
        LOG.debug("Permit denied for ugi: {} for method: {}",
            ugi, methodName);
        String msg =
            "Router " + router.getRouterId() +
                " is overloaded for NS: " + nsId;
        throw new StandbyException(msg);
      }
      if (rpcMonitor != null) {
        rpcMonitor.proxyOpPermitAccepted(nsId);
      }
      incrAcceptedPermitForNs(nsId);
    }
  }

  /**
   * Release permit for specific nsId after processing against downstream
   * nsId is completed.
   *  @param nsId Identifier of the block pool.
   * @param ugi UserGroupIdentifier associated with the user.
   * @param m Remote method that needs to be invoked.
   * @param controller fairness policy controller to release permit from
   */
  protected void releasePermit(final String nsId, final UserGroupInformation ugi,
      final RemoteMethod m, RouterRpcFairnessPolicyController controller) {

View on GitHub (pinned to 2add963021)

Solutions

1. Raise the per-nameservice handler quota (dfs.federation.router.handler.count per NS or async handler counts).
2. Scale out with additional Routers for the busy nameservice.
3. Investigate slow subcluster responses that tie up handler threads.

When it happens

Trigger: Per-nameservice overload control rejected the request because that NS's handler pool is saturated.

Common situations: One hot nameservice consuming all its allotted Router handlers during heavy client load.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/10a411a403f0eb1f. Report an issue: GitHub.