{"record":{"id":"3115c1ba0b7b487c","repo":"apache/hadoop","slug":"can-t-execute-the-shell-command-to-get-the-list-of","errorCode":null,"errorMessage":"Can't execute the shell command to get the list of group id for user '\" + userName + \"' (optionally + \" because of the command taking longer than the configured timeout: \" + timeout + \" seconds\")","messagePattern":"Can't execute the shell command to get the list of group id for user '\" \\+ userName \\+ \"' \\(optionally \\+ \" because of the command taking longer than the configured timeout: \" \\+ timeout \\+ \" seconds\"\\)","errorType":"exception","errorClass":"PartialGroupNameException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedUnixGroupsMapping.java","lineNumber":316,"sourceCode":"      try {\n        partialResolver.execute();\n        return parsePartialGroupNames(\n            groupNames, partialResolver.getOutput());\n      } catch (ExitCodeException ece) {\n        // If exception is thrown trying to get group id list,\n        // something is terribly wrong, so give up.\n        throw new PartialGroupNameException(\n            \"failed to get group id list for user '\" + userName + \"'\", ece);\n      } catch (IOException ioe) {\n        String message =\n            \"Can't execute the shell command to \" +\n            \"get the list of group id for user '\" + userName + \"'\";\n        if (partialResolver.isTimedOut()) {\n          message +=\n              \" because of the command taking longer than \" +\n              \"the configured timeout: \" + timeout + \" seconds\";\n        }\n        throw new PartialGroupNameException(message, ioe);\n      }\n    }\n  }\n\n  /**\n   * Split group names into a set.\n   *\n   * @param groupNames a string representing the user's group names\n   * @return a set of group names\n   */\n  @VisibleForTesting\n  protected Set<String> resolveFullGroupNames(String groupNames) {\n    StringTokenizer tokenizer =\n        new StringTokenizer(groupNames, Shell.TOKEN_SEPARATOR_REGEX);\n    Set<String> groups = new LinkedHashSet<>();\n    while (tokenizer.hasMoreTokens()) {\n      groups.add(tokenizer.nextToken());\n    }","sourceCodeStart":298,"sourceCodeEnd":334,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedUnixGroupsMapping.java#L298-L334","documentation":"Thrown by Hadoop's default shell-based group mapping during fallback resolution. After `groups <user>` exits non-zero but still printed names (some groups unresolvable), the mapper reruns `id -Gn <user>` to reconcile names with ids; if that second shell command raises a plain IOException that is not an exit-code error (notably a ShellCommandExecutor timeout), it is wrapped in PartialGroupNameException with this message. The timeout suffix appears only when the executor reports it was timed out; the limit comes from hadoop.security.groups.shell.command.timeout.","triggerScenarios":"ShellBasedUnixGroupsMapping.resolvePartialGroupNames: called when the first `groups <user>` command threw ExitCodeException with non-empty output; then the `id -Gn <user>` executor threw IOException - fork/exec failure, process killed, or runtime exceeding hadoop.security.groups.shell.command.timeout (default 0ms, i.e. no timeout).","commonSituations":"Slow NSS/LDAP/sssd group resolution making `id -Gn` blow past a small configured timeout; users with unresolvable GIDs; hardened container images without a working shell or with broken /etc/nsswitch.conf; NameNode/ResourceManager resolving groups for incoming RPC users.","solutions":["Raise hadoop.security.groups.shell.command.timeout in core-site.xml (it is a duration, e.g. 30s or 60000ms) on the node performing lookups","On that node, run `id -Gn <user>` and `groups <user>` as the Hadoop service user to reproduce the underlying OS failure and see how long it takes","Fix OS-level resolution so `id` returns fast: repair sssd/nscd/nsswitch.conf, chase DNS or LDAP timeouts","If lookups are inherently slow or flaky, switch hadoop.security.group.mapping to org.apache.hadoop.security.LdapGroupsMapping or a CompositeGroupsMapping with caching"],"exampleFix":"<!-- before -->\n<property>\n  <name>hadoop.security.groups.shell.command.timeout</name>\n  <value>1s</value>\n</property>\n<!-- after -->\n<property>\n  <name>hadoop.security.groups.shell.command.timeout</name>\n  <value>30s</value>\n</property>","handlingStrategy":"try-catch","validationCode":"// Pre-flight the same lookup the mapper performs, on the node that will do it\nString user = \"appuser\";\nProcess p = new ProcessBuilder(\"id\", \"-Gn\", user).start();\nboolean ok = p.waitFor(5, TimeUnit.SECONDS) && p.exitValue() == 0;\nif (!ok) throw new IllegalStateException(\"OS group lookup fails for \" + user);","typeGuard":null,"tryCatchPattern":"try {\n  Set<String> groups = ugi.getGroupsSet();\n} catch (IOException e) {\n  // PartialGroupNameException is private; match on message content for the timeout variant\n  if (e.getMessage() != null && e.getMessage().contains(\"configured timeout\")) {\n    LOG.warn(\"group lookup timed out for {}\", ugi.getUserName());\n  }\n  throw e;\n}","preventionTips":["Size hadoop.security.groups.shell.command.timeout above the p99 `id -Gn` latency measured on your slowest node","Keep sssd/nscd healthy and warm; slow first lookups are the usual timeout source","Prefer LdapGroupsMapping with caching for clusters with slow NSS","Monitor for 'PartialGroupNameException' in NameNode logs as an early warning"],"tags":["hadoop","security","group-mapping","shell","timeout","unix"],"backgroundTag":"user-group-lookup-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}