apache/hadoop · error · IOException

Can't get Master Kerberos principal for use as renewer

Error message

Can't get Master Kerberos principal for use as renewer

What it means

On Kerberos-enabled clusters, TokenCache.obtainTokensForNamenodesInternal obtains HDFS delegation tokens with the 'master' principal as renewer. The renewer comes from Master.getMasterPrincipal(conf): yarn.resourcemanager.principal in yarn mode, mapreduce.jobtracker.kerberos.principal in classic. If it resolves to empty and the filesystem host is not in mapreduce.job.hdfs-servers.token-renewal.exclude, job submission aborts with this IOException.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java:134

    }
    return false;
  }

  /**
   * get delegation token for a specific FS
   * @param fs
   * @param credentials
   * @param conf
   * @throws IOException
   */
  static void obtainTokensForNamenodesInternal(FileSystem fs,
      Credentials credentials, Configuration conf, String renewer)
      throws IOException {
    // RM skips renewing token with empty renewer
    String delegTokenRenewer = "";
    if (!isTokenRenewalExcluded(fs, conf)) {
      if (StringUtils.isEmpty(renewer)) {
        throw new IOException(
            "Can't get Master Kerberos principal for use as renewer");
      } else {
        delegTokenRenewer = renewer;
      }
    }

    mergeBinaryTokens(credentials, conf);

    final Token<?> tokens[] = fs.addDelegationTokens(delegTokenRenewer,
                                                     credentials);
    if (tokens != null) {
      for (Token<?> token : tokens) {
        LOG.info("Got dt for " + fs.getUri() + "; "+token);
      }
    }
  }

  private static void mergeBinaryTokens(Credentials creds, Configuration conf) {

View on GitHub (pinned to 2add963021)

Solutions

1. Set yarn.resourcemanager.principal (yarn) or mapreduce.jobtracker.kerberos.principal (classic) to the service principal in the submitting client's configuration
2. Confirm the principal value parses (SecurityUtil.getServerPrincipal expands _HOST) and the realm is correct
3. If the RM should not renew tokens for that filesystem, add its host to mapreduce.job.hdfs-servers.token-renewal.exclude
4. Ensure the submission tool loads the cluster's core/yarn/mapred site files

Example fix

// before: bare configuration, no security XML
Configuration conf = new Configuration(false);
// after
Configuration conf = new Configuration(true); // loads site files
conf.set("yarn.resourcemanager.principal", "rm/_HOST@EXAMPLE.COM");

if (UserGroupInformation.isSecurityEnabled()) {
  boolean isYarn = "yarn".equals(conf.get("mapreduce.framework.name", ""));
  String principal = isYarn
      ? conf.get("yarn.resourcemanager.principal")
      : conf.get("mapreduce.jobtracker.kerberos.principal");
  if (principal == null || principal.isEmpty()) {
    throw new IOException("RM/JT Kerberos principal not configured; cannot obtain delegation tokens");
  }
}

try {
  job.submit();
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("Master Kerberos principal")) {
    throw new IOException("Submit failed: set yarn.resourcemanager.principal in the client config", e);
  }
  throw e;
}

When it happens

Trigger: Security enabled but the RM (or JT) principal config is missing from the submitting client's configuration; a custom submission tool that builds a bare Configuration without loading *-site.xml; principal key present under a different property name than the framework mode in use.

Common situations: Oozie/Shell drivers or SDK clients constructed with new Configuration(true) but missing yarn-site.xml on the classpath; classic-vs-yarn mode mismatch (framework name set but the wrong principal property configured); test rigs submitting from outside the cluster without the cluster's security XML.

Related errors

• Server asks us to fall back to SIMPLE auth, but this client
• Kerberos required for secure registry access
• Fetch of delegation token failed
• Security is enabled but block access tokens (via dfs.block.a
• Security is enabled but block access tokens (via dfs.block.a

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/04c4e09e5fa434e0. Report an issue: GitHub.