apache/hadoop · error · java.io.IOException

Failed update mount table " + mount

Error message

Failed update mount table " + mount

What it means

The FedBalance switch step failed because the router's MountTableManager.updateMountTableEntry() RPC returned a failure status (response.getStatus() == false) instead of throwing. updateMountTableDestination() had already fetched and modified the entry (destination replaced by dstNs/dstPath) and treats the rejected update as a hard IOException, aborting the balance job before the mount table refresh.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/rbfbalance/MountTableProcedure.java:120

        RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_DEFAULT);
    InetSocketAddress routerSocket = NetUtils.createSocketAddr(address);
    RouterClient rClient = new RouterClient(routerSocket, conf);
    try {
      MountTableManager mountTable = rClient.getMountTableManager();

      MountTable originalEntry = getMountEntry(mount, mountTable);
      if (originalEntry == null) {
        throw new IOException("Mount table " + mount + " doesn't exist");
      } else {
        RemoteLocation remoteLocation =
            new RemoteLocation(dstNs, dstPath, mount);
        originalEntry.setDestinations(Arrays.asList(remoteLocation));
        UpdateMountTableEntryRequest updateRequest =
            UpdateMountTableEntryRequest.newInstance(originalEntry);
        UpdateMountTableEntryResponse response =
            mountTable.updateMountTableEntry(updateRequest);
        if (!response.getStatus()) {
          throw new IOException("Failed update mount table " + mount);
        }
        rClient.getMountTableManager().refreshMountTableEntries(
            RefreshMountTableEntriesRequest.newInstance());
      }
    } finally {
      rClient.close();
    }
  }

  /**
   * Gets the mount table entry.
   * @param mount name of the mount entry.
   * @param mountTable the mount table.
   * @return corresponding mount entry.
   * @throws IOException in case of failure to retrieve mount entry.
   */
  public static MountTable getMountEntry(String mount,
      MountTableManager mountTable)

View on GitHub (pinned to 2add963021)

Solutions

1. Check the router and state store (ZooKeeper/DB) logs at the failure timestamp for the underlying write error
2. Verify the mount entry's current state with hdfs dfsrouteradmin -list
3. Once the state store is healthy, retry the fedbalance job — the procedure re-executes the switch phase
4. Serialize admin changes to the same mount point so nothing races the procedure

// Retry the switch step with backoff; it is re-entrant after state store recovery
for (int attempt = 1; attempt <= 3; attempt++) {
  try {
    MountTableProcedure.updateMountTableDestination(mount, dstNs, dstPath, conf);
    break;
  } catch (IOException e) {
    if (attempt == 3 || !isTransient(e)) throw e;
    TimeUnit.SECONDS.sleep(5L * attempt);
  }
}

When it happens

Trigger: The router accepts the connection but the update fails server-side: state store (ZooKeeper/DB) write error or session loss, the mount record changed concurrently, router internal error while persisting the entry.

Common situations: ZooKeeper session expiry or state store hiccup exactly during the switch phase; a second admin operation editing the same mount entry at once; router restarting while the balance job hits its switch step.

Related errors

• Failed update mount table " + mount + " with readonly=" + re
• Mount table " + mount + " doesn't exist
• The mount point doesn't exist. path=" + fedPath
• The mount point has more than one destination. path={}
• State Store does not have an interface for {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/dc71abd49499af6f. Report an issue: GitHub.