apache/hadoop · error · IOException

Fetch of delegation token failed

Error message

Fetch of delegation token failed

What it means

HdfsDtFetcher is the HDFS DtFetcher plugin behind `hdfs fetchdt` token retrieval: it opens a FileSystem from the URL and calls fs.getDelegationToken(renewer). If that returns null - the FileSystem issued no token - it logs FETCH_FAILED and throws IOException('Fetch of delegation token failed').

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HdfsDtFetcher.java:78

  /**
   *  Returns Token object via FileSystem, null if bad argument.
   *  @param conf - a Configuration object used with FileSystem.get()
   *  @param creds - a Credentials object to which token(s) will be added
   *  @param renewer  - the renewer to send with the token request
   *  @param url  - the URL to which the request is sent
   *  @return a Token, or null if fetch fails.
   */
  public Token<?> addDelegationTokens(Configuration conf, Credentials creds,
                                  String renewer, String url) throws Exception {
    if (!url.startsWith(getServiceName().toString())) {
      url = getServiceName().toString() + "://" + url;
    }
    FileSystem fs = FileSystem.get(URI.create(url), conf);
    Token<?> token = fs.getDelegationToken(renewer);
    if (token == null) {
      LOG.error(FETCH_FAILED);
      throw new IOException(FETCH_FAILED);
    }
    creds.addToken(token.getService(), token);
    return token;
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Confirm the target cluster actually has security enabled (hadoop.security.authentication=kerberos) - without it no delegation token exists to fetch
2. Use a proper hdfs:// URL naming the HA nameservice or NameNode RPC address
3. Check for typos in the URL and that the FileSystem it resolves supports delegation tokens

Example fix

// before: cluster has no Kerberos, or URL is wrong
hdfs fetchdt --webservice http://wronghost:9870 /tmp/token
// after: security-enabled cluster, correct nameservice URL
hdfs fetchdt --webservice http://nn1:9870 /tmp/token   // nn1 in a kerberos-secured cluster

import org.apache.hadoop.security.SecurityUtil;

URI u = URI.create(url);
if (!"hdfs".equals(u.getScheme())) {
  throw new IllegalArgumentException("fetchdt expects an hdfs:// URL, got: " + url);
}
if (!SecurityUtil.getAuthenticationMethod(conf).equals(
    org.apache.hadoop.security.UserGroupInformation.AuthenticationMethod.KERBEROS)) {
  // no security -> no delegation tokens exist; skip the fetch rather than fail
  return;
}

try {
  fetcher.addDelegationTokens(conf, creds, renewer, url);
} catch (IOException e) {
  if (e.getMessage().contains("Fetch of delegation token failed")) {
    // token was not issued: verify security is enabled and the URL scheme,
    // then surface an actionable error to the operator
  } else {
    throw e;
  }
}

When it happens

Trigger: Fetching a delegation token from a URI whose FileSystem does not hand one out: a cluster running without security (getDelegationToken returns null when not Kerberos-enabled), a non-hdfs scheme, or a URL pointing at the wrong service. The null check at HdfsDtFetcher.java:78 is the throw site.

Common situations: Running `hdfs fetchdt --webservice <url> <file>` against a simple-auth cluster; typos in the URL; pointing fetchdt at webhdfs/file URLs whose implementations return null tokens; Oozie/CRON jobs fetching tokens for clusters without Kerberos.

Related errors

• Failed to create SecretManager
• Delegation Token can be issued only with kerberos or web aut
• Delegation Token can be renewed only with kerberos or web au
• No delegation token found for this identifier
• Can't load state from image in a running SecretManager.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/484cdb6e3fbedad7. Report an issue: GitHub.