apache/hadoop · error · NoSuchElementException

No more items in iterator

Error message

No more items in iterator

What it means

The listing iterator returned by listStatus(f, recursive) throws NoSuchElementException from next() when there are no more entries. This is standard Java iterator contract enforcement: next() was called after hasNext() returned false (either initially, or after fetching the next batch via listStatusBatch came back empty).

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java:434

    return new RemoteIterator<FileStatus>() {
      private DirectoryEntries entries = listStatusBatch(p, null);
      private int index = 0;

      @Override
      public boolean hasNext() {
        return index < entries.getEntries().length || entries.hasMore();
      }

      private void fetchMore() throws IOException {
        byte[] token = entries.getToken();
        entries = listStatusBatch(p, token);
        index = 0;
      }

      @Override
      public FileStatus next() throws IOException {
        if (!hasNext()) {
          throw new NoSuchElementException("No more items in iterator");
        } else {
          if (index == entries.getEntries().length) {
            fetchMore();
            if (!hasNext()) {
              throw new NoSuchElementException("No more items in iterator");
            }
          }

          return entries.getEntries()[index++];
        }
      }
    };
  }

  public static long dateToLong(final Date date) {
    return date == null ? 0L : date.getTime();
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Always guard next() with hasNext(): while (it.hasNext()) { ... it.next(); }
2. Prefer the array API listStatus(f) or Iterators.toArray(...) which handles exhaustion internally
3. If you already guard and still hit it (empty batch after hasMore()=true), report it as a paging bug in hadoop-tos with the token/batch details

Example fix

// before
Iterator<RawFileStatus> it = fs.listStatus(dir, true);
while (true) { RawFileStatus s = it.next(); process(s); } // NoSuchElementException at end

// after
while (it.hasNext()) { RawFileStatus s = it.next(); process(s); }

Iterator<RawFileStatus> it = fs.listStatus(dir, recursive);
while (it.hasNext()) {
  RawFileStatus s = it.next(); // only call next() when hasNext() just returned true
  process(s);
}

try { s = it.next(); }
catch (NoSuchElementException e) { break; } // last resort; prefer hasNext()

When it happens

Trigger: Calling it.next() without a preceding it.hasNext() check, or a loop like while(true) { it.next(); } on the iterator returned by RawFileSystem.listStatus(Path, boolean).

Common situations: Hand-rolled pagination loops that assume a fixed batch size; code converted from array-based listStatus() to the recursive iterator without preserving the hasNext() guard; streaming consumers that call next() then handle NoSuchElementException as flow control.

Related errors

• No more entry in {f}
• Listing '%s' failed
• listing object '%s' failed.
• Interrupted while listing using DFS, prefix={}, marker={}
• Can't open %s because it is a directory

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7e4fa2710ecf69a9. Report an issue: GitHub.