apache/hadoop · error · IOException

already exists

Error message

${tokenPath} already exists

What it means

Thrown by the JobHistoryServer's filesystem-backed recovery state store when storeToken tries to persist a delegation token whose target file (TOKEN_FILE_PREFIX + sequence number, sharded into bucket directories) already exists on the backing FileSystem. The store is only active when mapreduce.jobhistory.recovery.enable=true and the filesystem state store is configured, so this indicates the store already holds state for the same token sequence number. It almost always means a stale, reused, or shared recovery-store directory rather than a code defect.

Solutions

1. Verify exactly one active JobHistoryServer writes to the configured store URI (mapreduce.jobhistory.recovery.store.fs.uri).
2. If persisted token state is disposable, stop the JHS and remove/empty the recovery store directory; clients simply re-authenticate with newly issued tokens.
3. If state must be kept, list the bucket directories, locate the file named token_<sequence> from the message, and remove only that conflicting file.
4. Confirm mapreduce.jobhistory.recovery.enable / mapreduce.jobhistory.recovery.store.class are intentional and not left enabled from a previous test setup.

try {
  store.storeToken(tokenId, renewDate);
} catch (IOException e) {
  if (e.getMessage().endsWith("already exists")) {
    // duplicate token state in the recovery store: remove stale file or treat as idempotent
  } else {
    throw e;
  }
}

When it happens

Trigger: JHS restarts with recovery enabled against a store directory that still contains a token file with the same sequence number; two JobHistoryServer processes pointed at the same mapreduce.jobhistory.recovery.store.fs.uri; token sequence numbering reset (fresh secret manager) while old token files remain in the store.

Common situations: Redeploying or reinstalling JHS without cleaning the recovery store; sharing one HDFS store location between two JHS instances (test/prod overlap); restoring a store from backup that already contains the tokens about to be re-stored.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

• already exists
• Could not rename to
• Could not rename to
• Error loading token master key from
• Error loading token state from

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3e7be8c80b66a722. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryServerFileSystemStateStoreService.java:134

  @Override
  public HistoryServerState loadState() throws IOException {
    LOG.info("Loading history server state from " + rootStatePath);
    HistoryServerState state = new HistoryServerState();
    loadTokenState(state);
    return state;
  }

  @Override
  public void storeToken(MRDelegationTokenIdentifier tokenId,
      Long renewDate) throws IOException {
    if (LOG.isDebugEnabled()) {
      LOG.debug("Storing token " + tokenId.getSequenceNumber());
    }

    Path tokenPath = getTokenPath(tokenId);
    if (fs.exists(tokenPath)) {
      throw new IOException(tokenPath + " already exists");
    }

    createNewFile(tokenPath, buildTokenData(tokenId, renewDate));
  }

  @Override
  public void updateToken(MRDelegationTokenIdentifier tokenId,
      Long renewDate) throws IOException {
    if (LOG.isDebugEnabled()) {
      LOG.debug("Updating token " + tokenId.getSequenceNumber());
    }

    // Files cannot be atomically replaced, therefore we write a temporary
    // update file, remove the original token file, then rename the update
    // file to the token file. During recovery either the token file will be
    // used or if that is missing and an update file is present then the
    // update file is used.
    Path tokenPath = getTokenPath(tokenId);

View on GitHub (pinned to 2add963021)