apache/iceberg · warning

Failed to load catalog

Error message

Failed to load catalog: {}

What it means

A logged warning raised inside the catalogName resolver lambda passed to SparkUtil.catalogAndIdentifier when catalogManager.catalog(name) throws. Instead of failing identifier resolution outright, the resolver returns null so the next catalog/name-part interpretation is tried. It means one candidate catalog name could not be loaded (missing catalog config, bad class, init failure), and resolution falls through.

Solutions

1. Verify the catalog is configured: spark.sql.catalog.<name>=org.apache.iceberg.spark.SparkCatalog plus its type/uri options.
2. Fix the catalog name typo in the identifier or SQL statement.
3. Add the catalog implementation jar to the Spark classpath if the implementation class is missing.
4. Check the chained exception for the root cause (metastore connection, bad URI, invalid options).
5. Fully qualify identifiers with a configured catalog name so resolution does not guess name parts.

Example fix

// before
SELECT * FROM prod.db.tbl;  // 'prod' not configured
// after
spark.conf.set("spark.sql.catalog.prod", "org.apache.iceberg.spark.SparkCatalog");
spark.conf.set("spark.sql.catalog.prod.type", "hadoop");
SELECT * FROM prod.db.tbl;

// confirm the catalog is configured before referencing it
if (spark.conf().get("spark.sql.catalog.prod", null) == null) {
  throw new IllegalArgumentException("Catalog 'prod' is not configured");
}

// wrap identifier resolution and fall back to an explicit catalog reference
try { spark.table("prod.db.tbl"); } catch (Exception e) { spark.table("default.db.tbl"); }

When it happens

Trigger: Resolving a table/namespace identifier whose first name part is treated as a catalog name but CatalogManager fails to load it — e.g., 'spark.catalog.mynewcatalog' never configured, catalog implementation class missing from the classpath, or catalog init throwing.

Common situations: Typos in catalog names in SQL ('iceberg_tbl' vs catalog 'iceberg'); catalogs referenced in multi-part identifiers that were never configured in spark.sql.catalog.*; catalog plugin classes not shipped on the classpath; Hadoop/Hive metastore connection failures during catalog init.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Cannot use catalog ( ): not a TableCatalog
• Failed to load catalog
• Failed to load catalog
• Invalid session catalog:
• Invalid session catalog: sparkSessionCatalog

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b20fe9593dd1055d. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/Spark3Util.java:847

  public static CatalogAndIdentifier catalogAndIdentifier(
      SparkSession spark, List<String> nameParts, CatalogPlugin defaultCatalog) {
    CatalogManager catalogManager = spark.sessionState().catalogManager();

    String[] currentNamespace;
    if (defaultCatalog.equals(catalogManager.currentCatalog())) {
      currentNamespace = catalogManager.currentNamespace();
    } else {
      currentNamespace = defaultCatalog.defaultNamespace();
    }

    Pair<CatalogPlugin, Identifier> catalogIdentifier =
        SparkUtil.catalogAndIdentifier(
            nameParts,
            catalogName -> {
              try {
                return catalogManager.catalog(catalogName);
              } catch (Exception e) {
                LOG.warn("Failed to load catalog: {}", catalogName, e);
                return null;
              }
            },
            Identifier::of,
            defaultCatalog,
            currentNamespace);
    return new CatalogAndIdentifier(catalogIdentifier);
  }

  private static TableCatalog asTableCatalog(CatalogPlugin catalog) {
    if (catalog instanceof TableCatalog) {
      return (TableCatalog) catalog;
    }

    throw new IllegalArgumentException(
        String.format(
            "Cannot use catalog %s(%s): not a TableCatalog",
            catalog.name(), catalog.getClass().getName()));

View on GitHub (pinned to 86d9c8fc54)