apache/hadoop · error · PartialGroupNameException

Can't execute the shell command to get the list of group id

Error message

Can't execute the shell command to get the list of group id for user '" + userName + "' (optionally + " because of the command taking longer than the configured timeout: " + timeout + " seconds")

What it means

Thrown by Hadoop's default shell-based group mapping during fallback resolution. After `groups <user>` exits non-zero but still printed names (some groups unresolvable), the mapper reruns `id -Gn <user>` to reconcile names with ids; if that second shell command raises a plain IOException that is not an exit-code error (notably a ShellCommandExecutor timeout), it is wrapped in PartialGroupNameException with this message. The timeout suffix appears only when the executor reports it was timed out; the limit comes from hadoop.security.groups.shell.command.timeout.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ShellBasedUnixGroupsMapping.java:316

      try {
        partialResolver.execute();
        return parsePartialGroupNames(
            groupNames, partialResolver.getOutput());
      } catch (ExitCodeException ece) {
        // If exception is thrown trying to get group id list,
        // something is terribly wrong, so give up.
        throw new PartialGroupNameException(
            "failed to get group id list for user '" + userName + "'", ece);
      } catch (IOException ioe) {
        String message =
            "Can't execute the shell command to " +
            "get the list of group id for user '" + userName + "'";
        if (partialResolver.isTimedOut()) {
          message +=
              " because of the command taking longer than " +
              "the configured timeout: " + timeout + " seconds";
        }
        throw new PartialGroupNameException(message, ioe);
      }
    }
  }

  /**
   * Split group names into a set.
   *
   * @param groupNames a string representing the user's group names
   * @return a set of group names
   */
  @VisibleForTesting
  protected Set<String> resolveFullGroupNames(String groupNames) {
    StringTokenizer tokenizer =
        new StringTokenizer(groupNames, Shell.TOKEN_SEPARATOR_REGEX);
    Set<String> groups = new LinkedHashSet<>();
    while (tokenizer.hasMoreTokens()) {
      groups.add(tokenizer.nextToken());
    }

View on GitHub (pinned to 2add963021)

Solutions

  1. Raise hadoop.security.groups.shell.command.timeout in core-site.xml (it is a duration, e.g. 30s or 60000ms) on the node performing lookups
  2. On that node, run `id -Gn <user>` and `groups <user>` as the Hadoop service user to reproduce the underlying OS failure and see how long it takes
  3. Fix OS-level resolution so `id` returns fast: repair sssd/nscd/nsswitch.conf, chase DNS or LDAP timeouts
  4. If lookups are inherently slow or flaky, switch hadoop.security.group.mapping to org.apache.hadoop.security.LdapGroupsMapping or a CompositeGroupsMapping with caching

Example fix

<!-- before -->
<property>
  <name>hadoop.security.groups.shell.command.timeout</name>
  <value>1s</value>
</property>
<!-- after -->
<property>
  <name>hadoop.security.groups.shell.command.timeout</name>
  <value>30s</value>
</property>
Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-flight the same lookup the mapper performs, on the node that will do it
String user = "appuser";
Process p = new ProcessBuilder("id", "-Gn", user).start();
boolean ok = p.waitFor(5, TimeUnit.SECONDS) && p.exitValue() == 0;
if (!ok) throw new IllegalStateException("OS group lookup fails for " + user);

Try / catch

try {
  Set<String> groups = ugi.getGroupsSet();
} catch (IOException e) {
  // PartialGroupNameException is private; match on message content for the timeout variant
  if (e.getMessage() != null && e.getMessage().contains("configured timeout")) {
    LOG.warn("group lookup timed out for {}", ugi.getUserName());
  }
  throw e;
}

Prevention

When it happens

Trigger: ShellBasedUnixGroupsMapping.resolvePartialGroupNames: called when the first `groups <user>` command threw ExitCodeException with non-empty output; then the `id -Gn <user>` executor threw IOException - fork/exec failure, process killed, or runtime exceeding hadoop.security.groups.shell.command.timeout (default 0ms, i.e. no timeout).

Common situations: Slow NSS/LDAP/sssd group resolution making `id -Gn` blow past a small configured timeout; users with unresolvable GIDs; hardened container images without a working shell or with broken /etc/nsswitch.conf; NameNode/ResourceManager resolving groups for incoming RPC users.

Understand the failure class

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3115c1ba0b7b487c. Report an issue: GitHub.