apache/iceberg · error · org.apache.iceberg.flink.CatalogException

Failed to list partitions of table %s

Error message

Failed to list partitions of table %s

What it means

FlinkCatalog.listPartitions wraps IOException from Iceberg file scan planning (table.newScan().planFiles()) into a CatalogException with message "Failed to list partitions of table %s". Partition listing enumerates data-file partitions via a file-planning scan; if the underlying metadata/data files cannot be read (I/O failure), this exception is thrown with the table path and the original IOException as cause.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/FlinkCatalog.java:833

    Table table = loadIcebergTable(tablePath);

    if (table.spec().isUnpartitioned()) {
      throw new TableNotPartitionedException(icebergCatalog.name(), tablePath);
    }

    Set<CatalogPartitionSpec> set = Sets.newHashSet();
    try (CloseableIterable<FileScanTask> tasks = table.newScan().planFiles()) {
      for (DataFile dataFile : CloseableIterable.transform(tasks, FileScanTask::file)) {
        Map<String, String> map = Maps.newHashMap();
        StructLike structLike = dataFile.partition();
        PartitionSpec spec = table.specs().get(dataFile.specId());
        for (int i = 0; i < structLike.size(); i++) {
          map.put(spec.fields().get(i).name(), String.valueOf(structLike.get(i, Object.class)));
        }
        set.add(new CatalogPartitionSpec(map));
      }
    } catch (IOException e) {
      throw new CatalogException(
          String.format("Failed to list partitions of table %s", tablePath), e);
    }

    return Lists.newArrayList(set);
  }

  @Override
  public List<CatalogPartitionSpec> listPartitions(
      ObjectPath tablePath, CatalogPartitionSpec partitionSpec) throws CatalogException {
    throw new UnsupportedOperationException();
  }

  @Override
  public List<CatalogPartitionSpec> listPartitionsByFilter(
      ObjectPath tablePath, List<Expression> filters) throws CatalogException {
    throw new UnsupportedOperationException();
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Inspect the cause chain (e.getCause()) for the real I/O error (403, NoSuchKey, timeout) and fix storage access accordingly.
2. Refresh/validate storage credentials (FileIO configuration, IAM role, instance profile) and retry.
3. Retry the listing — planFiles is a read-only scan and transient network failures are common.
4. Check that referenced manifests/snapshots still exist (expireSnapshots misconfiguration can delete live metadata).
5. Verify FileIO/storage endpoint configuration (region, endpoint) matches the warehouse location.

Example fix

// before
List<CatalogPartitionSpec> parts = catalog.listPartitions(tablePath);

// after
try {
  List<CatalogPartitionSpec> parts = catalog.listPartitions(tablePath);
} catch (CatalogException e) {
  if (e.getCause() instanceof IOException) {
    // retry transient storage I/O failure
    parts = retryList(catalog, tablePath);
  } else {
    throw e;
  }
}

// pre-check storage reachability before listing
Table t = icebergTable(tablePath); Preconditions.checkArgument(t != null && !t.spec().isUnpartitioned());

try { return catalog.listPartitions(tablePath); } catch (CatalogException e) { if (e.getCause() instanceof IOException && isTransient(e.getCause())) { return retryList(tablePath); } throw e; }

When it happens

Trigger: Calling FlinkCatalog.listPartitions on a partitioned Iceberg table when planFiles() raises IOException — e.g. missing/corrupt manifest files, unreadable object store, credentials expired, network partition to HDFS/S3 during manifest read.

Common situations: S3/GCS/Azure credentials rotated or expired between table load and scan; metadata file deleted by retention job while a snapshot still references it; transient network failure to the storage layer; misconfigured FileIO (wrong endpoint/region).

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• RuntimeIOException
• Failed to plan files for main index
• Failed to load iceberg table from table loader: ${tableLoade
• Failed to create tableMaintenance
• Failed to load iceberg table from table loader: ${tableLoade

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/cacf5e7958f1fe67. Report an issue: GitHub.