apache/hadoop · error · IOException

Invalid UID, could not determine effective user

Error message

Invalid UID, could not determine effective user

What it means

During login, the JDK's UnixLoginModule can fail with 'invalid null input' when the OS cannot map the current process UID to a user (getpwuid found nothing). UGI detects that message and rethrows it as IOException('Invalid UID, could not determine effective user') because the failure is OS-level, not Kerberos.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java:2080

    try {
      HadoopLoginContext login = newLoginContext(
        authenticationMethod.getLoginAppName(), subject, loginConf);
      login.login();
      UserGroupInformation ugi = new UserGroupInformation(login.getSubject());
      // attach login context for relogin unless this was a pre-existing
      // subject.
      if (subject == null) {
        params.put(LoginParam.PRINCIPAL, ugi.getUserName());
        ugi.setLogin(login);
        ugi.setLastLogin(Time.now());
      }
      return ugi;
    } catch (LoginException le) {
      String msg = le.getMessage();
      if (msg != null && msg.contains("invalid null input")) {
        // This error from the JDK indicates that the OS couldn't map the UID of this process to an
        // actual user. Throw this as an IOException, because it's not related to Kerberos.
        throw new IOException(INVALID_UID, le);
      }
      KerberosAuthException kae =
        new KerberosAuthException(FAILURE_TO_LOGIN, le);
      if (params != null) {
        kae.setPrincipal(params.get(LoginParam.PRINCIPAL));
        kae.setKeytabFile(params.get(LoginParam.KEYTAB));
        kae.setTicketCacheFile(params.get(LoginParam.CCACHE));
      }
      throw kae;
    }
  }

  // parameters associated with kerberos logins.  may be extended to support
  // additional authentication methods.
  enum LoginParam {
    PRINCIPAL,
    KEYTAB,
    CCACHE,

View on GitHub (pinned to 2add963021)

Solutions

1. Give the runtime UID a passwd entry on the host: `useradd -o -u <uid> -g <gid> <name>` (or maintain it via LDAP/nss)
2. In containers, inject the user into /etc/passwd at entrypoint (echo "user:x:$(id -u):$(id -g)::/:/bin/bash" >> /etc/passwd)
3. Run the JVM as an existing registered user instead of an arbitrary UID
4. Verify with `getent passwd $(id -u)` on the node before starting Hadoop processes

Example fix

# before: container runs as unreferenced UID 1002321
# after: entrypoint ensures a passwd entry exists
echo "hadoop:x:$(id -u):$(id -g)::/home/hadoop:/bin/bash" >> /etc/passwd
exec "$@"

// Fail fast when the process UID has no passwd entry
if (java.nio.file.Files.lines(java.nio.file.Paths.get("/etc/passwd"))
        .noneMatch(l -> l.startsWith("^[^:]*:" + System.getProperty("user.name")))) {
  // simpler robust check below
}
String uid = String.valueOf(com.sun.security.auth.module.UnixSystem.class
    == null ? -1 : new com.sun.security.auth.module.UnixSystem().getUid());
java.util.Scanner s = new java.util.Scanner(Runtime.getRuntime()
    .exec(new String[]{"getent", "passwd", uid}).getInputStream())
    .useDelimiter("\\A");
if (!s.hasNext()) {
  throw new IllegalStateException("UID " + uid + " has no passwd entry");
}

try {
  UserGroupInformation.loginUserFromKeytab(principal, keytab);
} catch (IOException e) {
  if (e.getMessage() != null
      && e.getMessage().contains("Invalid UID")) {
    throw new IllegalStateException(
        "process UID is not mapped in /etc/passwd - add a passwd entry", e);
  }
  throw e;
}

When it happens

Trigger: doSubjectLogin/loginUserFromKeytab running in a process whose numeric UID has no entry in /etc/passwd (or the NSS passwd source) - typical of containers/schedulers launching JVMs as arbitrary unreferenced UIDs.

Common situations: Docker/Kubernetes pods running as a random numeric UID (OpenShift-style) without a passwd entry; nsswitch.conf not consulting the right passwd source; LDAP-managed passwd entries unreachable at login time.

Related errors

• Failed to find user in name " + subject
• Fewer lines of output than expected
• Can't execute the shell command to get the list of group id
• Illegal principal name " + name + ": " + ioe.toString()
• Problem with Kerberos auth_to_local name configuration

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5c4d6abb35685e06. Report an issue: GitHub.