{"record":{"id":"9a6d988797fa7d29","repo":"apache/hadoop","slug":"filesystem-not-generating-delegation-tokens-url","errorCode":null,"errorMessage":"Filesystem not generating Delegation Tokens: {url}","messagePattern":"Filesystem not generating Delegation Tokens: (.+?)","errorType":"exception","errorClass":"DelegationTokenIOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/S3ADtFetcher.java","lineNumber":75,"sourceCode":"  /**\n   *  Returns Token object via FileSystem, null if bad argument.\n   *  @param conf - a Configuration object used with FileSystem.get()\n   *  @param creds - a Credentials object to which token(s) will be added\n   *  @param renewer  - the renewer to send with the token request\n   *  @param url  - the URL to which the request is sent\n   *  @return a Token, or null if fetch fails.\n   */\n  public Token<?> addDelegationTokens(Configuration conf,\n      Credentials creds,\n      String renewer,\n      String url) throws Exception {\n    if (!url.startsWith(getServiceName().toString())) {\n      url = getServiceName().toString() + \"://\" + url;\n    }\n    FileSystem fs = FileSystem.get(URI.create(url), conf);\n    Token<?> token = fs.getDelegationToken(renewer);\n    if (token == null) {\n      throw new DelegationTokenIOException(FETCH_FAILED + \": \" + url);\n    }\n    creds.addToken(token.getService(), token);\n    return token;\n  }\n}\n","sourceCodeStart":57,"sourceCodeEnd":81,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/S3ADtFetcher.java#L57-L81","documentation":"S3ADtFetcher implements the 's3a' TokenFetcher used by tools like `hadoop fetchdt`: it opens the filesystem for the URL and calls getDelegationToken(renewer). S3AFileSystem returns null when delegation tokens are not enabled (fs.s3a.delegation.token.binding defaults to empty), so the fetcher fails with DelegationTokenIOException FETCH_FAILED 'Filesystem not generating Delegation Tokens: <url>'.","triggerScenarios":"Running a delegation-token fetch (hadoop fetchdt s3a://bucket, or a KeyingTool/TokenFetcher integration) against an S3A filesystem where fs.s3a.delegation.token.binding is unset, so fs.getDelegationToken(renewer) returns null.","commonSituations":"Cluster-wide core-site.xml has no delegation binding configured; the fetchdt client's config differs from the filesystem's; trying to collect s3a tokens for a distcp/Hive workflow before enabling DT support.","solutions":["Configure a delegation-token binding on the target filesystem: fs.s3a.delegation.token.binding=org.apache.hadoop.fs.s3a.auth.delegation.S3ATokenBinding (full credentials) or org.apache.hadoop.fs.s3a.auth.delegation.SessionTokenBinding (session credentials)","Make sure the fetchdt process uses the same configuration as the filesystem instance it contacts (same core-site, same bucket-level overrides)","Re-run the fetchdt command after the config change; the FS must be re-instantiated to pick it up"],"exampleFix":"<!-- before: DT support off (default) -->\n<!-- fs.s3a.delegation.token.binding unset -->\n\n<!-- after -->\n<property>\n  <name>fs.s3a.delegation.token.binding</name>\n  <value>org.apache.hadoop.fs.s3a.auth.delegation.S3ATokenBinding</value>\n</property>","handlingStrategy":"validation","validationCode":"String binding = conf.getTrimmed(\"fs.s3a.delegation.token.binding\", \"\");\nif (binding.isEmpty()) {\n  throw new IllegalStateException(\n      \"Cannot fetchdt s3a tokens: set fs.s3a.delegation.token.binding on the target filesystem\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  Token<?> t = new S3ADtFetcher().addDelegationTokens(conf, creds, renewer, \"s3a://bucket\");\n} catch (DelegationTokenIOException e) {\n  if (e.getMessage().startsWith(\"fetch failed\")) { // FETCH_FAILED\n    // DT not enabled on target FS: enable binding or skip token collection\n    LOG.error(\"Enable fs.s3a.delegation.token.binding on the S3A filesystem, then retry\");\n  }\n  throw e;\n}","preventionTips":["Enable fs.s3a.delegation.token.binding before adding s3a to fetchdt workflows","Use the same core-site.xml for the fetching client as the filesystem","Re-instantiate the filesystem after enabling the binding"],"tags":["aws","s3a","delegation-token","fetchdt","configuration"],"backgroundTag":"delegation-token-unavailable","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}