apache/iceberg · warning

Failed to load catalog: {}

Error message

Failed to load catalog: {}

What it means

When resolving a table/part identifier, Spark3Util asks the Spark CatalogManager to load each candidate catalog by name. If catalog loading throws, the exception is logged as a warning and null is returned so identifier resolution can continue with the next candidate or the default catalog. This means the named catalog is unusable, not necessarily that the table lookup failed.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java:864

  public static CatalogAndIdentifier catalogAndIdentifier(
      SparkSession spark, List<String> nameParts, CatalogPlugin defaultCatalog) {
    CatalogManager catalogManager = spark.sessionState().catalogManager();

    String[] currentNamespace;
    if (defaultCatalog.equals(catalogManager.currentCatalog())) {
      currentNamespace = catalogManager.currentNamespace();
    } else {
      currentNamespace = defaultCatalog.defaultNamespace();
    }

    Pair<CatalogPlugin, Identifier> catalogIdentifier =
        SparkUtil.catalogAndIdentifier(
            nameParts,
            catalogName -> {
              try {
                return catalogManager.catalog(catalogName);
              } catch (Exception e) {
                LOG.warn("Failed to load catalog: {}", catalogName, e);
                return null;
              }
            },
            Identifier::of,
            defaultCatalog,
            currentNamespace);
    return new CatalogAndIdentifier(catalogIdentifier);
  }

  private static TableCatalog asTableCatalog(CatalogPlugin catalog) {
    if (catalog instanceof TableCatalog) {
      return (TableCatalog) catalog;
    }

    throw new IllegalArgumentException(
        String.format(
            "Cannot use catalog %s(%s): not a TableCatalog",
            catalog.name(), catalog.getClass().getName()));

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Check the chained exception in the log — fix the catalog config (e.g. uri, warehouse, credential options)
  2. Verify the catalog name spelling and that it is registered in Spark session catalog confs
  3. Test catalog availability directly (e.g. SHOW CATALOGS / listing via the catalog client) to isolate connectivity issues

Example fix

// before
spark.sql.catalog.my_cat.type=rest
spark.sql.catalog.my_cat.uri=https://wrong-host:8181
// after
spark.sql.catalog.my_cat.type=rest
spark.sql.catalog.my_cat.uri=http://catalog-host:8181
Defensive patterns

Strategy: try-catch

Validate before calling

try {
  Catalog c = spark.sessionState().catalogManager().catalog(name);
} catch (Exception e) {
  // catalog misconfigured; fix conf before resolving identifiers
}

Try / catch

try {
  spark.table("my_cat.db.tbl");
} catch (Exception e) {
  LOG.warn("Catalog load failed", e);
  // check conf: spark.sql.catalog.my_cat.*
}

Prevention

When it happens

Trigger: catalogManager.catalog(name) throws during catalogAndIdentifier resolution — e.g. the catalog plugin class cannot be instantiated, its options are wrong, or the underlying metastore is unreachable.

Common situations: Typos in catalog name in USE/queries; misconfigured catalog options in Spark conf (missing warehouse/metadata URLs); metastore (Hive/REST/Nessie) downtime.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/90a2c40fbb3177fb. Report an issue: GitHub.