apache/hadoop · error · FileNotFoundException

The object key %s is a invalid key, detail: %s

Error message

The object key %s is a invalid key, detail: %s

What it means

getFileStatus() converts ParentNotDirectoryException into FileNotFoundException, preserving the original message ('The object key %s is a invalid key ...'). So this message surfacing as FileNotFoundException means the path sits under an existing file (invalid key in directory-bucket terms): existence checks treat 'under a file' as 'does not exist', matching Hadoop-exists() semantics.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java:505

          break;
        } else {
          throw new FileAlreadyExistsException(String.format("Can't make directory for path '%s',"
                  + " it is a file.", parent));
        }
      } catch (FileNotFoundException ignored) {
      }
      parent = parent.getParent();
    } while (parent != null);
  }

  @Override
  public FileStatus getFileStatus(Path path) throws IOException {
    try {
      return innerFileStatus(path);
    } catch (ParentNotDirectoryException e) {
      // Treat ParentNotDirectoryException as FileNotFoundException for the case that check whether
      // path exist or not.
      throw new FileNotFoundException(e.getMessage());
    }
  }


  /**
   * Get the file status of given path.
   *
   * @param f the path
   * @return {@link RawFileStatus} describe file status info.
   * @throws FileNotFoundException       if the path doesn't exist.
   * @throws ParentNotDirectoryException if the path is locating under an existing file, which is
   *                                     not allowed in directory bucket case.
   */
  RawFileStatus innerFileStatus(Path f) throws ParentNotDirectoryException, FileNotFoundException {
    Path qualifiedPath = f.makeQualified(uri, workingDir);
    RawFileStatus fileStatus = getFileStatusOrNull(qualifiedPath);
    if (fileStatus == null) {
      throw new FileNotFoundException(

View on GitHub (pinned to 2add963021)

Solutions

1. Fix the key layout: never address children of a file object; use sibling prefixes instead
2. Remove or rename the file occupying the parent position, then retry
3. If probing existence, expect FileNotFoundException here and treat it as 'not present'

Example fix

// before
FileStatus st = fs.getFileStatus(new Path("/logs/app.txt/2024")); // '/logs/app.txt' is a file

// after: keep objects and prefixes disjoint
FileStatus st = fs.getFileStatus(new Path("/logs/app.txt-2024")); // sibling, not child

// existence probe that also tolerates under-a-file paths
FileStatus st;
try { st = fs.getFileStatus(p); }
catch (FileNotFoundException e) { st = null; } // includes parent-is-file case

try { st = fs.getFileStatus(p); }
catch (FileNotFoundException e) {
  if (e.getMessage() != null && e.getMessage().contains("is a invalid key")) {
    // path descends from a file object: fix the key layout
  } else { /* ordinary missing path */ }
}

When it happens

Trigger: fs.getFileStatus('/logs/app.txt/child') where '/logs/app.txt' exists as a file object; fs.exists() and globStatus on such paths funnel through getFileStatus and return false / empty rather than throw elsewhere.

Common situations: Probing a path whose prefix collides with a previously written file; recovery tools scanning for '/file/part' style keys after a layout change; code that assumes missing parents are silently treated as directories like on HDFS.

Related errors

• Can't open %s because it is a directory
• {} is a directory
• {} already exists
• Not supported
• Failed to rename since it is prohibited to rename dest path

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/028b3705795f96c3. Report an issue: GitHub.