apache/iceberg · info

Unsupported runtime filter

Error message

Unsupported runtime filter {}

What it means

A WARN logged by SparkCopyOnWriteScan.filter when a Spark V2 runtime Predicate (pushed by the engine, e.g. from a DynamicFileFilter or shuffle-origin filter) cannot be converted or applied to narrow down the pre-planned tasks. The scan falls back to the unfiltered task set, so the query still returns correct results but loses the runtime-filter optimization.

Solutions

  1. Verify the runtime filter targets a column type/partition field Iceberg can convert
  2. Check the Iceberg/Spark version pair for supported runtime-filter kinds
  3. Ignore safely if performance is acceptable — results remain correct
Defensive patterns

Strategy: fallback

Prevention

When it happens

Trigger: filter() receives a runtime predicate whose kind/name is not one SparkV2Filters.convert or the local filtering logic supports, or filterFilesOnMetadata is not applicable for the predicate type.

Common situations: Engine pushes a runtime filter on a non-partition or expression type Iceberg does not map; using Spark features that generate unsupported runtime predicates; upgrading Spark and encountering new predicate kinds.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/04860b9215d0adbd. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkCopyOnWriteScan.java:121

        // so filter files only if it is beneficial
        if (filteredLocations == null || fileLocations.size() < filteredLocations.size()) {
          this.filteredLocations = fileLocations;
          List<FileScanTask> filteredTasks =
              tasks().stream()
                  .filter(file -> fileLocations.contains(file.file().location()))
                  .collect(Collectors.toList());

          LOG.info(
              "{} of {} task(s) for table {} matched runtime file filter with {} location(s)",
              filteredTasks.size(),
              tasks().size(),
              table().name(),
              fileLocations.size());

          resetTasks(filteredTasks);
        }
      } else {
        LOG.warn("Unsupported runtime filter {}", predicate);
      }
    }
  }

  @Override
  public boolean equals(Object o) {
    if (this == o) {
      return true;
    }

    if (o == null || getClass() != o.getClass()) {
      return false;
    }

    SparkCopyOnWriteScan that = (SparkCopyOnWriteScan) o;
    return table().name().equals(that.table().name())
        && Objects.equals(table().uuid(), that.table().uuid())
        && Objects.equals(snapshot, that.snapshot)

View on GitHub (pinned to 86d9c8fc54)