{"record":{"id":"5ab2d200396ab29c","repo":"apache/hadoop","slug":"failed-to-unmarshall-token-for-canonicaluri","errorCode":null,"errorMessage":"Failed to unmarshall token for {canonicalUri}","messagePattern":"Failed to unmarshall token for (.+?)","errorType":"exception","errorClass":"DelegationTokenIOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/S3ADelegationTokens.java","lineNumber":592,"sourceCode":"      throws IOException {\n\n    checkArgument(token != null, \"null token\");\n    AbstractS3ATokenIdentifier identifier;\n    // harden up decode beyond that Token does itself\n    try {\n      identifier = token.decodeIdentifier();\n    } catch (RuntimeException e) {\n      Throwable cause = e.getCause();\n      if (cause != null) {\n        // its a wrapping around class instantiation.\n        throw new DelegationTokenIOException(\"Decoding S3A token \" + cause,\n            cause);\n      } else {\n        throw e;\n      }\n    }\n    if (identifier == null) {\n      throw new DelegationTokenIOException(\"Failed to unmarshall token for \"\n          + getCanonicalUri());\n    }\n    identifier.validate();\n    return identifier;\n  }\n\n  /**\n   * Return a string for use in building up the User-Agent field, so\n   * get into the S3 access logs. Useful for diagnostics.\n   * Delegates to {{@link AbstractDelegationTokenBinding#getUserAgentField()}}\n   * for the current binding.\n   * @return a string for the S3 logs or \"\" for \"nothing to add\"\n   */\n  public String getUserAgentField() {\n    return tokenBinding.getUserAgentField();\n  }\n\n  /**","sourceCodeStart":574,"sourceCodeEnd":610,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/S3ADelegationTokens.java#L574-L610","documentation":"If Token.decodeIdentifier() returns null instead of throwing (which Hadoop's Token does when the kind maps to no known identifier factory or decoding yields nothing), S3ADelegationTokens treats it as an unmarshalling failure and throws DelegationTokenIOException 'Failed to unmarshall token for <canonicalUri>' rather than NPE later. The canonical URI in the message identifies which filesystem the unusable token belongs to.","triggerScenarios":"A token of an unknown/unregistered kind (or with an empty payload) is present in the user's credentials for the s3a service, and decodeTokenIdentifier processes it during binding.","commonSituations":"Tokens from other S3A-compatible systems or old versions left in a credentials file; a token kind registered by a binding that is no longer on the classpath; partially written token stores.","solutions":["Remove the token for that s3a URI from the credentials (new Credentials for the job, or cancel the token) and let the filesystem deploy unbonded","Ensure the delegation-token binding that issued the token is on the classpath and configured on consumers","Re-fetch a fresh token after aligning versions"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"if (token.decodeIdentifier() == null) {\n  credentials.removeToken(token.getService());\n  LOG.warn(\"Token kind {} for {} cannot be decoded; removed\", token.getKind(), token.getService());\n}","typeGuard":null,"tryCatchPattern":"try {\n  tokens.bindToAnyDelegationToken();\n} catch (DelegationTokenIOException e) {\n  if (e.getMessage().contains(\"Failed to unmarshall token\")) {\n    credentials.removeToken(new Text(canonicalUri)); // unknown-kind token: purge and rebind\n    tokens.bindToAnyDelegationToken();\n  } else {\n    throw e;\n  }\n}","preventionTips":["Remove foreign/legacy tokens from credentials stores when changing bindings","Keep the issuing binding on the consumer classpath","Periodically purge credential files of tokens for decommissioned services"],"tags":["aws","s3a","delegation-token","deserialization","token-decode"],"backgroundTag":"token-decode-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}