apache/hadoop · error · DelegationTokenIOException

Token mismatch: expected token for {service} of type {kind}

Error message

Token mismatch: expected token for {service} of type {kind} but got a token of type {tokenKind}

What it means

When selecting a delegation token from the user's credentials, S3ADelegationTokens looks up the token for the s3a service and requires its Text kind to equal the kind of the configured binding. If a token exists for that URI but with a different kind (issued by a different S3A delegation mechanism), it throws DelegationTokenIOException with the TOKEN_MISMATCH prefix instead of silently using the wrong token.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/delegation/S3ADelegationTokens.java:638

  public static Token<AbstractS3ATokenIdentifier> lookupToken(
      final Credentials credentials,
      final Text service,
      final Text kind)
      throws DelegationTokenIOException {

    LOG.debug("Looking for token for service {} in credentials", service);
    Token<?> token = credentials.getToken(service);
    if (token != null) {
      Text tokenKind = token.getKind();
      LOG.debug("Found token of kind {}", tokenKind);
      if (kind.equals(tokenKind)) {
        // the Oauth implementation catches and logs here; this one
        // throws the failure up.
        return (Token<AbstractS3ATokenIdentifier>) token;
      } else {

        // there's a token for this URI, but its not the right DT kind
        throw new DelegationTokenIOException(
            DelegationTokenIOException.TOKEN_MISMATCH + ": expected token"
            + " for " + service
            + " of type " + kind
            + " but got a token of type " + tokenKind);
      }
    }
    // A token for the service was not found
    LOG.debug("No token for {} found", service);
    return null;
  }

  /**
   * Look up any token from the service; cast it to one of ours.
   * @param credentials credentials
   * @param service service to look up
   * @return any token found or null if none was
   * @throws ClassCastException if the token is of a wrong type.
   */

View on GitHub (pinned to 2add963021)

Solutions

1. Remove or cancel the mismatched token for that s3a URI, then rebind or re-fetch under the current binding
2. Keep the binding configuration identical across token issuer and all consumers
3. Give each job its own Credentials object instead of sharing UGI credentials across jobs with different bindings

Text service = new Text(fs.getCanonicalUri());
Token<?> existing = ugi.getCredentials().getToken(service);
if (existing != null && !expectedKind.equals(existing.getKind())) {
  ugi.getCredentials().removeToken(service);
  LOG.warn("Removed mismatched token kind {} for {}", existing.getKind(), service);
}

static boolean tokenHasKind(Token<?> t, Text expectedKind) {
  return t != null && expectedKind.equals(t.getKind());
}

try {
  tokens.bindToAnyDelegationToken();
} catch (DelegationTokenIOException e) {
  if (e.getMessage().startsWith(DelegationTokenIOException.TOKEN_MISMATCH)) {
    credentials.removeToken(service); // token from another binding: purge, then rebind
    tokens.bindToAnyDelegationToken();
  } else {
    throw e;
  }
}

When it happens

Trigger: selectToken(service, kind) finds a token whose getKind() differs from the binding's kind. E.g. credentials hold a token with kind of one binding while fs.s3a.delegation.token.binding selects another binding for the same s3a://bucket URI.

Common situations: Changing fs.s3a.delegation.token.binding (e.g. from SessionTokenBinding to S3ATokenBinding or vice versa) while old tokens remain in the credentials; a workflow where one component issues tokens under a different binding than the engine consuming them.

Related errors

• Delegation token is wrong class; expected a token identifier
• {component}: No AWS credentials
• {component}: Invalid AWS credentials in {credentials} requir
• No URI in {this}
• Incompatible EncryptionSecrets version: {versionId}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/60ea01bc8852346b. Report an issue: GitHub.