What causes "No more entries in {}" in apache/hadoop?

Gate every next() with hasNext(): while (it.hasNext()) { LocatedFileStatus s = it.next(); ... }

apache/hadoop · error · NoSuchElementException

No more entries in {}

Error message

No more entries in {}

What it means

NoSuchElementException thrown by the anonymous RemoteIterator<LocatedFileStatus> returned from FileSystem.listLocatedStatus(Path) (listFiles builds on it). The whole listing is materialized once via listStatus(f, filter) into stats; next() checks hasNext() and, once the cursor i reaches stats.length, throws with the queried path in the message. It is a plain Iterator-contract violation by the caller, not an IO failure.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:2292

   * @throws FileNotFoundException if <code>f</code> does not exist
   * @throws IOException if any I/O error occurred
   */
  protected RemoteIterator<LocatedFileStatus> listLocatedStatus(final Path f,
      final PathFilter filter)
  throws FileNotFoundException, IOException {
    return new RemoteIterator<LocatedFileStatus>() {
      private final FileStatus[] stats = listStatus(f, filter);
      private int i = 0;

      @Override
      public boolean hasNext() {
        return i<stats.length;
      }

      @Override
      public LocatedFileStatus next() throws IOException {
        if (!hasNext()) {
          throw new NoSuchElementException("No more entries in " + f);
        }
        FileStatus result = stats[i++];
        // for files, use getBlockLocations(FileStatus, int, int) to avoid
        // calling getFileStatus(Path) to load the FileStatus again
        BlockLocation[] locs = result.isFile() ?
            getFileBlockLocations(result, 0, result.getLen()) :
            null;
        return new LocatedFileStatus(result, locs);
      }
    };
  }

  /**
   * Generic iterator for implementing {@link #listStatusIterator(Path)}.
   */
  protected class DirListingIterator<T extends FileStatus> implements
      RemoteIterator<T> {

View on GitHub (pinned to 2add963021)

Solutions

Gate every next() with hasNext(): while (it.hasNext()) { LocatedFileStatus s = it.next(); ... }
If you need lookahead, wrap the iterator and buffer one element yourself instead of probing with next()
Do not reuse a drained RemoteIterator; request a fresh one from listLocatedStatus/listFiles

Example fix

// before
RemoteIterator<LocatedFileStatus> it = fs.listFiles(dir, true);
while (true) {
  LocatedFileStatus s = it.next(); // NoSuchElementException on empty dir
  process(s);
}

// after
RemoteIterator<LocatedFileStatus> it = fs.listFiles(dir, true);
while (it.hasNext()) {
  process(it.next());
}

Defensive patterns

Strategy: validation

Validate before calling

RemoteIterator<LocatedFileStatus> it = fs.listFiles(dir, true);
while (it.hasNext()) {          // mandatory guard before every next()
  LocatedFileStatus st = it.next();
  process(st);
}

Prevention

Never call next() outside a while(it.hasNext()) block
Treat empty directories as normal: check hasNext() before the first next()
Create a new iterator per traversal pass; a drained RemoteIterator cannot be reset

When it happens

Trigger: Calling next() without a preceding true hasNext(): while(true){ it.next(); }, calling next() after the loop already consumed the final entry, or unconditionally doing LocatedFileStatus s = it.next() on a possibly-empty directory listing.

Common situations: Custom InputFormat/split calculators or Spark/Hive readers adapted from java.util.Iterator code; assuming a directory is non-empty; reusing an iterator after it was drained by an earlier loop. Note the file list is fetched up front, so concurrent deletion does not cause this — it is purely caller logic.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d46ba761cefa7e1f. Report an issue: GitHub.