apache/iceberg · warning

Unable to list metadata directory

Error message

Unable to list metadata directory {}

What it means

HadoopCatalog.isTableDir logs this WARN when listing a table's metadata directory for version-hint/metadata files fails with an IOException that is classified as a permission error. The method returns false (treats the directory as not a table) instead of throwing, which can silently hide tables from listTables results.

Solutions

1. Grant the catalog/execution user read+execute permissions on the table metadata directories.
2. Fix HDFS ACL/Kerberos or S3 IAM policies so listing is allowed.
3. If the suppression is undesirable, adjust shouldSuppressPermissionError configuration to surface the error instead of returning false.

Example fix

// before (HDFS)
hdfs dfs -chmod -R 750 /warehouse/db/table
// after
hdfs dfs -chmod -R 755 /warehouse/db/table  # or grant the service account via setfacl

// check access before listing
// hdfs dfs -test -d /warehouse/db/table/metadata
// or in code: fs.access(metadataPath, FsAction.READ_EXECUTE)

try {
  catalog.listTables(Namespace.of("db"));
} catch (UncheckedIOException e) {
  LOG.error("metadata dir not listable: {}", e.getCause());
}

When it happens

Trigger: listTables or isNamespace walks candidate directories; fs.listStatus on <table>/metadata throws an access-denied IOException and shouldSuppressPermissionError(e) is true (fs.permissions.suppress.* config or default behavior).

Common situations: HDFS/S3 ACLs limiting the listing user; Kerberos/permission misconfiguration; catalog listing running under a service account without read access to some namespaces.

Understand the failure class

Background: Permission denied / not authorized / 403 Forbidden: access-control rejections when the caller lacks the required role, grant, or ownership — this error's family across 18 libraries.

Related errors

• Unable to list directory
• Create namespace failed
• Error reading version hint file
• Error trying to recover the latest version number for
• Failed to create file

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/35dc00403682256e. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopCatalog.java:167

    if (suppressPermissionError) {
      return ioException instanceof AccessDeniedException
          || (ioException.getMessage() != null
              && ioException.getMessage().contains("AuthorizationPermissionMismatch"));
    }
    return false;
  }

  private boolean isTableDir(Path path) {
    Path metadataPath = new Path(path, "metadata");
    // Only the path which contains metadata is the path for table, otherwise it could be
    // still a namespace.
    try {
      return fs.listStatus(metadataPath, TABLE_FILTER).length >= 1;
    } catch (FileNotFoundException e) {
      return false;
    } catch (IOException e) {
      if (shouldSuppressPermissionError(e)) {
        LOG.warn("Unable to list metadata directory {}", metadataPath, e);
        return false;
      } else {
        throw new UncheckedIOException(e);
      }
    }
  }

  private boolean isDirectory(Path path) {
    try {
      return fs.getFileStatus(path).isDirectory();
    } catch (FileNotFoundException e) {
      return false;
    } catch (IOException e) {
      if (shouldSuppressPermissionError(e)) {
        LOG.warn("Unable to list directory {}", path, e);
        return false;
      } else {
        throw new UncheckedIOException(e);

View on GitHub (pinned to 86d9c8fc54)