What causes "Unexpected file in store: ${dir}" in apache/hadoop?

Inspect the reported path: hadoop fs -ls on its parent to see the offending file.

apache/hadoop · error · FileAlreadyExistsException

Unexpected file in store: ${dir}

Error message

Unexpected file in store: ${dir}

What it means

During state-store initialization the service creates its directory layout (root, keys directory, token bucket directories). createDir expects any pre-existing path to be a directory; if getFileStatus reports a regular file, it throws FileAlreadyExistsException 'Unexpected file in store'. The configured store location is occupied by a file where the store must create and own a directory.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryServerFileSystemStateStoreService.java:232

    return tokenId.getSequenceNumber() % NUM_TOKEN_BUCKETS;
  }

  private Path getTokenBucketPath(int bucketId) {
    return new Path(tokenStatePath,
        String.format(TOKEN_BUCKET_NAME_FORMAT, bucketId));
  }

  private Path getTokenPath(MRDelegationTokenIdentifier tokenId) {
    Path bucketPath = getTokenBucketPath(getBucketId(tokenId));
    return new Path(bucketPath,
        TOKEN_FILE_PREFIX + tokenId.getSequenceNumber());
  }

  private void createDir(Path dir) throws IOException {
    try {
      FileStatus status = fs.getFileStatus(dir);
      if (!status.isDirectory()) {
        throw new FileAlreadyExistsException("Unexpected file in store: "
            + dir);
      }
      if (!status.getPermission().equals(DIR_PERMISSIONS)) {
        fs.setPermission(dir, DIR_PERMISSIONS);
      }
    } catch (FileNotFoundException e) {
      fs.mkdirs(dir, DIR_PERMISSIONS);
    }
  }

  private void createNewFile(Path file, byte[] data)
      throws IOException {
    Path tmp = new Path(file.getParent(), TMP_FILE_PREFIX + file.getName());
    writeFile(tmp, data);
    try {
      if (!fs.rename(tmp, file)) {
        throw new IOException("Could not rename " + tmp + " to " + file);
      }

View on GitHub (pinned to 2add963021)

Solutions

Inspect the reported path: hadoop fs -ls on its parent to see the offending file.
Remove or relocate the file.
Re-point the store URI to an empty dedicated directory and restart JHS.

Defensive patterns

Strategy: validation

Validate before calling

# verify the store root is a directory and its parent is writable before JHS start
hadoop fs -test -d /jhs-state || hadoop fs -mkdir -p /jhs-state
hadoop fs -test -w / || echo 'parent not writable'

Prevention

Provision the store path as an empty dedicated directory.
Do not place marker files or artifacts inside the recovery store tree.
Validate the store URI in deployment scripts before starting JHS.

When it happens

Trigger: mapreduce.jobhistory.recovery.store.fs.uri points at a path occupied by a plain file; a file (not a directory) named like the keys or a bucket directory exists inside the store tree; provisioning scripts created a marker file instead of a directory.

Common situations: Store URI typo resolving to a file; artifact copy or touch commands run inside the store path; manual manipulation of the store tree.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/cc40fffe42f6de26. Report an issue: GitHub.