apache/druid · critical · ISE

Failed to authenticate user principal [%s] with keytab [%s]

Error message

Failed to authenticate user principal [%s] with keytab [%s]

What it means

HdfsStorageAuthentication.authenticate() performs Kerberos login via UserGroupInformation.loginUserFromKeytab(principal, keytab). Any IOException during login is wrapped in this ISE naming the principal and keytab. It means Kerberos authentication for HDFS access failed.

Source

Thrown at extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsStorageAuthentication.java:70

   * In case of any bug fix make sure to fix the code in JobHelper#authenticate as well.
   */
  @LifecycleStart
  public void authenticate()
  {
    String principal = hdfsKerberosConfig.getPrincipal();
    String keytab = hdfsKerberosConfig.getKeytab();
    if (!Strings.isNullOrEmpty(principal) && !Strings.isNullOrEmpty(keytab)) {
      UserGroupInformation.setConfiguration(hadoopConf);
      if (UserGroupInformation.isSecurityEnabled()) {
        try {
          if (UserGroupInformation.getCurrentUser().hasKerberosCredentials() == false
              || !UserGroupInformation.getCurrentUser().getUserName().equals(principal)) {
            log.info("Trying to authenticate user [%s] with keytab [%s]..", principal, keytab);
            UserGroupInformation.loginUserFromKeytab(principal, keytab);
          }
        }
        catch (IOException e) {
          throw new ISE(e, "Failed to authenticate user principal [%s] with keytab [%s]", principal, keytab);
        }
      }
    }
  }

  @LifecycleStop
  public void stop()
  {
    //noop
  }
}

View on GitHub (pinned to 9b90983fd2)

Solutions

Verify the keytab file exists and is readable by the Druid user on every node
Confirm the principal in the keytab exactly matches the configured principal (use klist -kt keytab)
Test manually with kinit -kt <keytab> <principal> and check krb5.conf / KDC reachability and clock sync
Replace the keytab if rotated/expired and restart the node

Example fix

// before (wrong path)
druid.hadoop.security.kerberos.principal=druid/_HOST@EXAMPLE.COM
druid.hadoop.security.kerberos.keytab=/etc/druid/conf/old.keytab
// after
druid.hadoop.security.kerberos.principal=druid/_HOST@EXAMPLE.COM
druid.hadoop.security.kerberos.keytab=/etc/security/keytabs/druid.keytab

Defensive patterns

Strategy: validation

Validate before calling

File keytabFile = new File(keytabPath);
if (!keytabFile.canRead()) throw new IllegalStateException("Keytab missing/unreadable: " + keytabPath);
// verify principal with: klist -kt keytab

Type guard

null

Try / catch

try {
  hdfsAuth.authenticate();
} catch (IllegalStateException e) {
  log.error("Kerberos login failed; check keytab/principal/KDC", e);
  // alert and stop instead of continuing with anonymous HDFS access
  throw e;
}

Prevention

Distribute and rotate keytabs on all nodes with correct ownership/permissions
Test kinit -kt <keytab> <principal> outside Druid before startup
Keep krb5.conf and clock sync (NTP) correct; monitor KDC reachability

When it happens

Trigger: authenticate(principal, keytab) is called (during lifecycle start) and loginUserFromKeytab throws: keytab file missing/unreadable, wrong principal name not present in the keytab, invalid/expired keytab, or Kerberos KDC unreachable / clock skew.

Common situations: Wrong druid.hadoop.security.kerberos.principal/keytab paths; keytab not distributed to all nodes; hostname mismatch between principal and host; missing krb5.conf; expired keytabs after rotation.

Understand the failure class

Authentication and authorization failures — expired tokens, bad credentials, and missing scopes.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/eccf5defeedb70ee. Report an issue: GitHub.