{"record":{"id":"2864a6e592d28e6d","repo":"apache/hadoop","slug":"failed-update-mount-table-mount-with-reado","errorCode":null,"errorMessage":"Failed update mount table \" + mount + \" with readonly=\" + readOnly","messagePattern":"Failed update mount table \" \\+ mount \\+ \" with readonly=\" \\+ readOnly","errorType":"exception","errorClass":"java.io.IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/rbfbalance/MountTableProcedure.java","lineNumber":202,"sourceCode":"      Configuration conf) throws IOException {\n    String address = conf.getTrimmed(RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_KEY,\n        RBFConfigKeys.DFS_ROUTER_ADMIN_ADDRESS_DEFAULT);\n    InetSocketAddress routerSocket = NetUtils.createSocketAddr(address);\n    RouterClient rClient = new RouterClient(routerSocket, conf);\n    try {\n      MountTableManager mountTable = rClient.getMountTableManager();\n\n      MountTable originalEntry = getMountEntry(mount, mountTable);\n      if (originalEntry == null) {\n        throw new IOException(\"Mount table \" + mount + \" doesn't exist\");\n      } else {\n        originalEntry.setReadOnly(readOnly);\n        UpdateMountTableEntryRequest updateRequest =\n            UpdateMountTableEntryRequest.newInstance(originalEntry);\n        UpdateMountTableEntryResponse response =\n            mountTable.updateMountTableEntry(updateRequest);\n        if (!response.getStatus()) {\n          throw new IOException(\n              \"Failed update mount table \" + mount + \" with readonly=\"\n                  + readOnly);\n        }\n        rClient.getMountTableManager().refreshMountTableEntries(\n            RefreshMountTableEntriesRequest.newInstance());\n      }\n    } finally {\n      rClient.close();\n    }\n  }\n\n  @Override\n  public void write(DataOutput out) throws IOException {\n    super.write(out);\n    Text.writeString(out, mount);\n    Text.writeString(out, dstPath);\n    Text.writeString(out, dstNs);\n    conf.write(out);","sourceCodeStart":184,"sourceCodeEnd":220,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/rbfbalance/MountTableProcedure.java#L184-L220","documentation":"setMountReadOnly() in the FedBalance procedure submitted an UpdateMountTableEntryRequest with the readonly flag set, but the router's MountTableManager.updateMountTableEntry() returned a failure status, so the procedure throws IOException('Failed update mount table <mount> with readonly=<readOnly>') and the balance job aborts. This is the state-store-side rejection of the readonly toggle, not a missing entry.","triggerScenarios":"The update RPC returns status=false: state store write failure (ZooKeeper/DB unreachable or session expired), concurrent modification of the same mount entry, or router-internal error while persisting the readonly change.","commonSituations":"Transient state store outage during the switch phase; concurrent dfsrouteradmin -update on the same mount; router under restart while the balance toggles readonly.","solutions":["Inspect router and state store logs for the update failure cause","Re-check the mount's current readonly state: hdfs dfsrouteradmin -list","After state store recovery, retry the fedbalance job (the switch phase is re-entrant)","Prevent concurrent admin edits to the mount during balance operations"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Retry the readonly toggle after transient state store failures\nboolean done = false;\nfor (int attempt = 1; attempt <= 3 && !done; attempt++) {\n  try {\n    MountTableProcedure.setMountReadOnly(mount, true, conf);\n    done = true;\n  } catch (IOException e) {\n    LOG.warn(\"readonly toggle attempt {} failed: {}\", attempt, e.getMessage());\n    if (attempt == 3) throw e;\n    TimeUnit.SECONDS.sleep(5L);\n  }\n}","preventionTips":["Monitor state store health during balance jobs","Serialize admin operations against the same mount entry","Re-check mount state with dfsrouteradmin -list before retrying"],"tags":["hadoop","rbf","fedbalance","mount-table","readonly","state-store","update-failed"],"backgroundTag":"admin-api-update-rejected","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}