apache/iceberg · warning

Unable to list directory

Error message

Unable to list directory {}

What it means

HadoopCatalog.isDirectory logs this WARN when fs.getFileStatus(path) throws an IOException classified as a permission error while checking whether a namespace path is a directory. The method returns false instead of throwing, so namespaces that cannot be stat'ed are silently skipped during listing operations.

Solutions

1. Grant the calling user read/execute on the namespace directory path.
2. Correct HDFS ACLs or S3 IAM/bucket policy to allow stat operations.
3. Tune the permission-error suppression setting if these errors should be fatal instead of skipped.

Example fix

// before
hdfs dfs -ls /warehouse/otherdb  # Permission denied
// after
sudo hdfs dfs -chown -R catalog_user /warehouse/otherdb
# or: hdfs dfs -setfacl -R -m user:catalog_user:r-x /warehouse/otherdb

// verify namespace dir accessibility before catalog operations
// hadoop fs -test -d /warehouse/db && hadoop fs -ls /warehouse/db

try {
  catalog.listTables(namespace);
} catch (UncheckedIOException e) {
  LOG.error("Cannot stat namespace dir: {}", e.getCause());
}

When it happens

Trigger: listTables or isNamespace calls isDirectory on a database/path; the filesystem returns an access-denied IOException and suppression of permission errors is enabled.

Common situations: Namespace directory owned by another user/group; missing execute bit on parent directories in HDFS; S3 bucket policy denying HEAD/GetObject on the prefix.

Understand the failure class

Background: Permission denied / not authorized / 403 Forbidden: access-control rejections when the caller lacks the required role, grant, or ownership — this error's family across 18 libraries.

Related errors

• Unable to list metadata directory
• Create namespace failed
• Error reading version hint file
• Error trying to recover the latest version number for
• Failed to create file

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/4a2f2ece21a1d7e3. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopCatalog.java:182

      return false;
    } catch (IOException e) {
      if (shouldSuppressPermissionError(e)) {
        LOG.warn("Unable to list metadata directory {}", metadataPath, e);
        return false;
      } else {
        throw new UncheckedIOException(e);
      }
    }
  }

  private boolean isDirectory(Path path) {
    try {
      return fs.getFileStatus(path).isDirectory();
    } catch (FileNotFoundException e) {
      return false;
    } catch (IOException e) {
      if (shouldSuppressPermissionError(e)) {
        LOG.warn("Unable to list directory {}", path, e);
        return false;
      } else {
        throw new UncheckedIOException(e);
      }
    }
  }

  @Override
  public List<TableIdentifier> listTables(Namespace namespace) {
    Preconditions.checkArgument(
        namespace.levels().length >= 1, "Missing database in table identifier: %s", namespace);

    Path nsPath = new Path(warehouseLocation, SLASH.join(namespace.levels()));
    Set<TableIdentifier> tblIdents = Sets.newHashSet();

    try {
      if (!isDirectory(nsPath)) {
        throw new NoSuchNamespaceException("Namespace does not exist: %s", namespace);

View on GitHub (pinned to 86d9c8fc54)