{"record":{"id":"2a78a73821c23f80","repo":"apache/hadoop","slug":"disallowed-rpc-access-from-at-not-listed-in","errorCode":null,"errorMessage":"Disallowed RPC access from {} at {}. Not listed in dfs.cluster.administrators","messagePattern":"Disallowed RPC access from (.+?) at (.+?)\\. Not listed in dfs\\.cluster\\.administrators","errorType":"exception","errorClass":"AccessControlException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSZKFailoverController.java","lineNumber":234,"sourceCode":"      LOG.error(\"DFSZKFailOverController exiting due to earlier exception \"\n          + t);\n      terminate(1, t);\n    }\n  }\n\n  @Override\n  protected void checkRpcAdminAccess() throws IOException, AccessControlException {\n    UserGroupInformation ugi = UserGroupInformation.getCurrentUser();\n    UserGroupInformation zkfcUgi = UserGroupInformation.getLoginUser();\n    if (adminAcl.isUserAllowed(ugi) ||\n        ugi.getShortUserName().equals(zkfcUgi.getShortUserName())) {\n      LOG.info(\"Allowed RPC access from \" + ugi + \" at \" + Server.getRemoteAddress());\n      return;\n    }\n    String msg = \"Disallowed RPC access from \" + ugi + \" at \" +\n        Server.getRemoteAddress() + \". Not listed in \" + DFSConfigKeys.DFS_ADMIN; \n    LOG.warn(msg);\n    throw new AccessControlException(msg);\n  }\n\n  /**\n   * capture local NN's thread dump and write it to ZKFC's log.\n   */\n  private void getLocalNNThreadDump() {\n    isThreadDumpCaptured = false;\n    // We use the same timeout value for both connection establishment\n    // timeout and read timeout.\n    int httpTimeOut = conf.getInt(\n        DFSConfigKeys.DFS_HA_ZKFC_NN_HTTP_TIMEOUT_KEY,\n        DFSConfigKeys.DFS_HA_ZKFC_NN_HTTP_TIMEOUT_KEY_DEFAULT);\n    if (httpTimeOut == 0) {\n      // If timeout value is set to zero, the feature is turned off.\n      return;\n    }\n    try {\n      String stacksUrl = DFSUtil.getInfoServer(localNNTarget.getAddress(),","sourceCodeStart":216,"sourceCodeEnd":252,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSZKFailoverController.java#L216-L252","documentation":"The ZKFC daemon exposes a small admin RPC server (ZKFCProtocol, contacted for example by 'hdfs haadmin -failover' graceful failover and other admin operations). checkRpcAdminAccess authorizes each call: the caller's UGI must pass adminAcl (built from dfs.cluster.administrators) or the caller's short username must equal the ZKFC daemon's login user. Otherwise the call is refused with AccessControlException and a WARN line naming the rejected ugi and remote address.","triggerScenarios":"Running 'hdfs haadmin -failover ...' as a user that is neither in the dfs.cluster.administrators ACL nor the same user as the one running the zkfc process; an ACL that lists groups/users with wrong syntax so adminAcl does not match the caller.","commonSituations":"Operators running failover as their personal account while zkfc runs as hdfs; dfs.cluster.administrators configured on the NameNodes but not picked up by the ZKFC's config; kerberos-authenticated admin whose short name differs from the daemon user.","solutions":["Run the admin command as the same user that runs the zkfc daemon (typically hdfs), e.g. 'sudo -u hdfs hdfs haadmin -failover nn1 nn2'.","Add the invoking user or group to dfs.cluster.administrators in hdfs-site.xml (format: comma-separated users, space, comma-separated groups) on the NameNode/zkfc hosts and restart zkfc.","Check the zkfc log line 'Disallowed RPC access from <ugi>' to see exactly which ugi and address the server saw, then align the ACL with it.","Verify with 'hdfs getconf -confKey dfs.cluster.administrators' on the zkfc host that the ACL actually loaded."],"exampleFix":"# before: run as operator, zkfc runs as hdfs\nhdfs haadmin -ns mycluster -failover nn1 nn2   # AccessControlException\n\n# after: same user as the zkfc daemon\nsudo -u hdfs hdfs haadmin -ns mycluster -failover nn1 nn2\n\n# or in hdfs-site.xml on the zkfc hosts, then restart zkfc\n<property>\n  <name>dfs.cluster.administrators</name>\n  <value>hdfs,opsadmin hadmin</value>\n</property>","handlingStrategy":"try-catch","validationCode":"// Before issuing graceful failover, confirm the operator passes the ACL\nString acl = conf.get(\"dfs.cluster.administrators\", \"\");\nString user = UserGroupInformation.getCurrentUser().getShortUserName();\nString zkfcUser = UserGroupInformation.getLoginUser().getShortUserName();\nif (!user.equals(zkfcUser) && !aclContains(acl, user)) {\n  throw new SecurityException(\"Run as \" + zkfcUser\n      + \" or add \" + user + \" to dfs.cluster.administrators\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  haAdmin.failover(fromNode, toNode);\n} catch (AccessControlException ace) {\n  // Message contains 'Not listed in dfs.cluster.administrators'\n  throw new UnsupportedOperationException(\n      \"Failover denied for \" + UserGroupInformation.getCurrentUser()\n          + \": add user to dfs.cluster.administrators or run as the zkfc daemon user\", ace);\n}","preventionTips":["Run failover and other ZKFC admin operations as the same OS user that runs the zkfc daemon.","Maintain dfs.cluster.administrators in config management and reload/restart zkfc after changes.","Include a periodic 'hdfs haadmin -getServiceState' from the monitoring user to surface ACL breaks before a real failover."],"tags":["hadoop","hdfs","access-control","acl","zkfc","permissions"],"backgroundTag":"acl-access-denied","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}