apache/iceberg · error · java.lang.UnsupportedOperationException

SparkCachedTableCatalog does not support table invalidation

Error message

SparkCachedTableCatalog does not support table invalidation

What it means

Capability guard in SparkCachedTableCatalog.invalidateTable: cached-table loads are resolved per lookup from the underlying catalog, so there is no cache entry to invalidate and the operation is refused. The message names the catalog class; Spark should direct invalidation at the real catalog.

Source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkCachedTableCatalog.java:91

    Preconditions.checkArgument(
        table.snapshotId() == null, "Cannot time travel based on both table identifier and AS OF");
    return table.copyWithSnapshotId(Long.parseLong(version));
  }

  @Override
  public SparkTable loadTable(Identifier ident, long timestampMicros) throws NoSuchTableException {
    SparkTable table = load(ident);
    Preconditions.checkArgument(
        table.snapshotId() == null, "Cannot time travel based on both table identifier and AS OF");
    // Spark passes microseconds but Iceberg uses milliseconds for snapshots
    long timestampMillis = TimeUnit.MICROSECONDS.toMillis(timestampMicros);
    long snapshotId = SnapshotUtil.snapshotIdAsOfTime(table.table(), timestampMillis);
    return table.copyWithSnapshotId(snapshotId);
  }

  @Override
  public void invalidateTable(Identifier ident) {
    throw new UnsupportedOperationException(CLASS_NAME + " does not support table invalidation");
  }

  @Override
  public SparkTable createTable(
      Identifier ident, StructType schema, Transform[] partitions, Map<String, String> properties)
      throws TableAlreadyExistsException {
    throw new UnsupportedOperationException(CLASS_NAME + " does not support creating tables");
  }

  @Override
  public SparkTable alterTable(Identifier ident, TableChange... changes) {
    throw new UnsupportedOperationException(CLASS_NAME + " does not support altering tables");
  }

  @Override
  public boolean dropTable(Identifier ident) {
    throw new UnsupportedOperationException(CLASS_NAME + " does not support dropping tables");
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

Remove/re-cache the cached table (UNCACHE TABLE) instead of invalidating it
Refresh via the underlying real catalog's invalidateTable
Update Iceberg — newer versions route invalidation through the table cache (SparkTableCache) instead of throwing

Example fix

// before
spark.catalog().invalidateTable("spark_catalog_cached.db.tbl"); // throws
// after
spark.sql("UNCACHE TABLE cached_tbl");
spark.catalog().invalidateTable("spark_catalog.db.tbl");

Defensive patterns

Strategy: fallback

Validate before calling

if (catalog instanceof SparkCachedTableCatalog) { invalidateViaRealCatalog(ident); return; }

Try / catch

try { catalog.invalidateTable(ident); } catch (UnsupportedOperationException e) { spark.sql("UNCACHE TABLE " + name); realCatalog.invalidateTable(ident); }

Prevention

Invalidate via the real catalog; refresh cached tables with UNCACHE/RECACHE
Upgrade Iceberg versions where invalidation is routed through SparkTableCache
Do not assume all TableCatalog implementations support invalidation

When it happens

Trigger: Calling invalidateTable(ident) on SparkCachedTableCatalog, e.g. when Spark's cache invalidation machinery refreshes tables after writes, or direct calls via the TableCatalog interface.

Common situations: Mixing cached-table references with normal writes and expecting auto-refresh; Spark's CatalogImpl.invalidateTable walking all registered catalogs including the cached one.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/947398933b915021. Report an issue: GitHub.