apache/hadoop · error · IOException

Same delegation token being added twice: {formatTokenId(iden

Error message

Same delegation token being added twice: {formatTokenId(identifier)}

What it means

Error "Same delegation token being added twice: {formatTokenId(identifier)}" thrown in apache/hadoop.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java:483

      DelegationKey dKey = allKeys.get(keyId);
      byte[] password = null;
      if (dKey == null) {
        LOG.warn("No KEY found for persisted identifier, expiring stored token " + formatTokenId(
            identifier));
        // make sure the token is expired
        renewDate = 0L;
      } else {
        password = createPassword(identifier.getBytes(), dKey.getKey());
      }
      if (identifier.getSequenceNumber() > getDelegationTokenSeqNum()) {
        setDelegationTokenSeqNum(identifier.getSequenceNumber());
      }
      if (getTokenInfo(identifier) == null) {
        currentTokens.put(identifier, new DelegationTokenInformation(renewDate, password,
            getTrackingIdIfEnabled(identifier)));
        addTokenForOwnerStats(identifier);
      } else {
        throw new IOException("Same delegation token being added twice: " +
            formatTokenId(identifier));
      }
    } finally {
      this.apiLock.writeLock().unlock();
    }
  }

  /** 
   * Update the current master key 
   * This is called once by startThreads before tokenRemoverThread is created, 
   * and only by tokenRemoverThread afterwards.
   */
  private void updateCurrentKey() throws IOException {
    LOG.info("Updating the current master key for generating delegation tokens");
    /* Create a new currentKey with an estimated expiry date. */
    int newCurrentId;
    newCurrentId = incrementCurrentKeyId();
    DelegationKey newKey = new DelegationKey(newCurrentId, System

View on GitHub (pinned to 2add963021)

Solutions

1. The same delegation token ({formatTokenId(identifier)}) is being added twice, which indicates duplicate recovery/replay from the store. Check the secret manager store for duplicates and clear stale entries, or deduplicate before adding.

When it happens

Trigger: Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java:483 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/80126bde4b238272. Report an issue: GitHub.