apache/hadoop · error · FileAlreadyExistsException

${keyPath} already exists

Error message

${keyPath} already exists

What it means

The filesystem state store persists each delegation master key as its own file (TOKEN_MASTER_KEY_FILE_PREFIX + keyId) under the token keys directory. storeTokenMasterKey throws FileAlreadyExistsException when that file already exists, meaning the store already contains a master key with the same keyId. Like the duplicate-token error, it signals stale state in the recovery store or a second writer rather than an intrinsic Hadoop fault.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryServerFileSystemStateStoreService.java:185

  @Override
  public void removeToken(MRDelegationTokenIdentifier tokenId)
      throws IOException {
    if (LOG.isDebugEnabled()) {
      LOG.debug("Removing token " + tokenId.getSequenceNumber());
    }
    deleteFile(getTokenPath(tokenId));
  }

  @Override
  public void storeTokenMasterKey(DelegationKey key) throws IOException {
    if (LOG.isDebugEnabled()) {
      LOG.debug("Storing master key " + key.getKeyId());
    }

    Path keyPath = new Path(tokenKeysStatePath,
        TOKEN_MASTER_KEY_FILE_PREFIX + key.getKeyId());
    if (fs.exists(keyPath)) {
      throw new FileAlreadyExistsException(keyPath + " already exists");
    }

    ByteArrayOutputStream memStream = new ByteArrayOutputStream();
    DataOutputStream dataStream = new DataOutputStream(memStream);
    try {
      key.write(dataStream);
      dataStream.close();
      dataStream = null;
    } finally {
      IOUtils.cleanupWithLogger(LOG, dataStream);
    }

    createNewFile(keyPath, memStream.toByteArray());
  }

  @Override
  public void removeTokenMasterKey(DelegationKey key)
      throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

1. Confirm a single JHS instance owns the store location.
2. If master-key state can be discarded, stop JHS and clear the token keys area of the recovery store.
3. Otherwise remove only the conflicting key file (TOKEN_MASTER_KEY_<keyId>) and restart JHS.
4. Ensure the secret manager always starts from the same recovery store so key ids keep advancing instead of resetting.

try {
  store.storeTokenMasterKey(key);
} catch (org.apache.hadoop.fs.FileAlreadyExistsException e) {
  // key file already in store: stale state or second writer; clean or skip
}

When it happens

Trigger: JHS restarted against a store that already contains a master key with the same key id; key id sequence reset while old key files remain; two JHS instances sharing one store URI both storing new master keys.

Common situations: Recovery directory reused across reinstalls; store restored from a snapshot; duplicated store location in HA-like deployments; experimentation leftover files in the keys directory.

Related errors

• ${tokenPath} already exists
• Could not rename ${tmp} to ${tokenPath}
• Unexpected file in store: ${dir}
• Could not rename ${tmp} to ${file}
• Unable to delete ${file}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a00378e3da27e089. Report an issue: GitHub.