apache/iceberg · warning

Exception listing files for

Error message

Exception listing files for {} at {}

What it means

Warning from the Flink maintenance ListFileSystemFiles operator when listing files in a directory for orphan-file detection throws. The exception is side-output to the DeleteOrphanFiles ERROR_STREAM and counted, so the action degrades: some files may be missed and, depending on failure mode, no orphans are reported for that location.

Solutions

1. Read the side-output ERROR_STREAM to inspect the wrapped exception and react
2. Verify the location URI (scheme/authority) matches the configured filesystem and equalSchemes()/equalAuthorities() settings
3. Check read permissions on the listed directory
4. Retry the job if the error was transient (concurrent deletion, throttling)

// Pre-check the location is listable
FileSystem fs = new Path(location).getFileSystem(conf);
fs.exists(new Path(location));

try { listFiles(location); } catch (Exception e) { LOG.warn("Exception listing files for {} at {}", location, ts, e); errorCounter.inc(); }

When it happens

Trigger: processElement lists a location via FileSystem.listFiles; an unhandled Exception (FileNotFound on a just-deleted dir, permission errors, IOException from object store) hits the catch block.

Common situations: Conflicting URI schemes/authorities between table metadata and configured locations; directory deleted concurrently; S3 throttling; missing read permission on the warehouse path.

Related errors

• Exception listing files for
• Exception listing files for
• Could not list sub directories, reached maximum depth:
• Exception planning scan for
• Exception planning scan for

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/7b957aaae096279c. Report an issue: GitHub.

Appendix: source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/ListFileSystemFiles.java:122

            "Cannot use prefix listing with FileIO %s which does not support prefix operations.",
            io);

        FileSystemWalker.listDirRecursivelyWithFileIO(
            (SupportsPrefixOperations) io, location, specs, predicate, out::collect);
      } else {
        Predicate<FileStatus> predicate = file -> file.getModificationTime() < olderThanTimestamp;
        FileSystemWalker.listDirRecursivelyWithHadoop(
            location,
            specs,
            predicate,
            configuration,
            Integer.MAX_VALUE,
            Integer.MAX_VALUE,
            dir -> {},
            out::collect);
      }
    } catch (Exception e) {
      LOG.warn("Exception listing files for {} at {}", location, ctx.timestamp(), e);
      ctx.output(DeleteOrphanFiles.ERROR_STREAM, e);
      errorCounter.inc();
    }
  }

  @Override
  public void close() throws Exception {
    super.close();
    tableLoader.close();
  }
}

View on GitHub (pinned to 86d9c8fc54)