apache/hadoop · error · FileNotFoundException

File {} does not exist in pseudo local file system

Error message

File {} does not exist in pseudo local file system

What it means

Thrown by PseudoLocalFs.validateFileNameFormat() when a path is not a valid pseudo-local file name. PseudoLocalFs is a virtual, read-only FileSystem used by gridmix that generates random data on the fly; every file URI must look like pseudo:///<name>.<fileSize> where the last dot-separated segment is a non-negative long giving the file size in bytes. The check fails when the URI scheme is not 'pseudo' or when Long.parseLong() of the last '.'-segment throws NumberFormatException or yields a negative value.

Source

Thrown at hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/PseudoLocalFs.java:134

   * @throws FileNotFoundException
   */
  long validateFileNameFormat(Path path) throws FileNotFoundException {
    path = this.makeQualified(path);
    boolean valid = true;
    long fileSize = 0;
    if (!path.toUri().getScheme().equals(getUri().getScheme())) {
      valid = false;
    } else {
      String[] parts = path.toUri().getPath().split("\\.");
      try {
        fileSize = Long.parseLong(parts[parts.length - 1]);
        valid = (fileSize >= 0);
      } catch (NumberFormatException e) {
        valid = false;
      }
    }
    if (!valid) {
      throw new FileNotFoundException("File " + path
          + " does not exist in pseudo local file system");
    }
    return fileSize;
  }

  /**
   * @See create(Path) for details
   */
  @Override
  public FSDataInputStream open(Path path, int bufferSize) throws IOException {
    long fileSize = validateFileNameFormat(path);
    InputStream in = new RandomInputStream(fileSize, bufferSize);
    return new FSDataInputStream(in);
  }

  /**
   * @See create(Path) for details
   */

View on GitHub (pinned to 2add963021)

Solutions

  1. Format the path as <uniqueName>.<sizeInBytes> (e.g. pseudo:///file0.1048576), or better, call PseudoLocalFs.generateFilePath(fileId, fileSize) which builds the correct relative path
  2. Make sure the path is qualified with the pseudo:// scheme that PseudoLocalFs.getUri() returns (URI.create("pseudo:///"))
  3. Verify the size suffix is a non-negative long before opening; validate with the same split("\\.") + Long.parseLong logic
  4. If you intended a real file, use the local FileSystem or HDFS instead of PseudoLocalFs

Example fix

// before
Path p = new Path("pseudo:///part-00000");
fs.open(p); // FileNotFoundException: no size suffix

// after
Path p = PseudoLocalFs.generateFilePath("part-00000", 1024L * 1024L);
// => pseudo:///part-00000.1048576
fs.open(p);
Defensive patterns

Strategy: validation

Validate before calling

// Validate a pseudo-local path before open()
static long pseudoFileSize(Path p) {
  if (!"pseudo".equals(p.toUri().getScheme())) {
    throw new IllegalArgumentException("Not a pseudo:// path: " + p);
  }
  String[] parts = p.toUri().getPath().split("\\.");
  long size = Long.parseLong(parts[parts.length - 1]);
  if (size < 0) throw new IllegalArgumentException("Negative size: " + p);
  return size;
}
// Or simply build paths only via PseudoLocalFs.generateFilePath(fileId, size)

Try / catch

try {
  fs.open(path);
} catch (FileNotFoundException e) {
  // path failed pseudo-local naming rules: scheme != pseudo or missing .<size> suffix
  LOG.warn("Invalid pseudo-local path {}", path, e);
}

Prevention

When it happens

Trigger: Calling open(path), getFileStatus(path), or create(path) on a PseudoLocalFs instance with: (1) a path whose last '.'-separated component is not a number (e.g. pseudo:///input.txt), (2) a negative size suffix (pseudo:///f.-5), (3) a path qualified with a different scheme (file:///f.1024) after makeQualified(), or (4) a path with no dot at all, where split returns the whole name and parse fails.

Common situations: Building gridmix/DistributedCacheEmulator input paths by hand instead of using PseudoLocalFs.generateFilePath(fileId, fileSize); passing raw job-trace paths into code that expects pseudo-local files; forgetting the '.<bytes>' size suffix; copying a plain filename from another FileSystem into pseudo:///. Also thrown (wrapped as 'File creation failed for <path>') from create().

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5a5c2d9f4a321e5e. Report an issue: GitHub.