apache/iceberg · error · IllegalStateException

Too many matching keys: more than

Error message

Too many matching keys: more than %d

What it means

ScanSummary's bounded size map (used for top-partition metrics) throws IllegalStateException when the number of distinct matching keys exceeds maxSize and the summary was configured with throwIfLimited=true, i.e. limiting results by dropping keys is not allowed.

Solutions

1. Increase the maxSize limit for the summary computation.
2. Construct the summary with throwIfLimited=false to truncate to the top keys instead of failing.
3. Narrow the scan (filter by partition/predicate) to reduce distinct matching keys.

Example fix

// before
new ScanSummary.LimitedSizeMap<>(maxSize, true); // throws when exceeded
// after
new ScanSummary.LimitedSizeMap<>(maxSize, false); // truncate to top keys

long distinctPartitions = estimateDistinctPartitions(table);
if (distinctPartitions > maxSize) { /* raise maxSize or disable throwIfLimited */ }

try { metrics = computeTopPartitionMetrics(...); } catch (IllegalStateException e) { /* increase maxSize or truncate instead of throwing */ }

When it happens

Trigger: Computing ScanSummary partition metrics on a table with more distinct partition keys than the configured maxSize while throwIfLimited is set.

Common situations: High-cardinality partitions (e.g. hourly partitions over years) with a small summary limit; summarizing unpartitioned-turned-partitioned large tables.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Cannot filter timestamps using predicate:
• Cannot satisfy time filters: time range may include expired…
• No timestamps can match filters:
• ALTER VIEW AS is not supported. Use CREATE OR REPLACE VIEW…
• apply(value) is deprecated, use bind(Type).apply(value)

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/344b20f0710ae589. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/ScanSummary.java:350

      this.maxSize = maxSize;
      this.throwIfLimited = throwIfLimited;
      this.map = Maps.newTreeMap(keyComparator);
      this.keyComparator = keyComparator;
    }

    public void update(K key, Function<V, V> updateFunc) {
      // if there is a cut and it comes before the given key, do nothing
      if (cut != null && keyComparator.compare(cut, key) <= 0) {
        return;
      }

      // call the update function and add the result to the map
      map.put(key, updateFunc.apply(map.get(key)));

      // enforce the size constraint and update the cut if some keys are excluded
      while (map.size() > maxSize) {
        if (throwIfLimited) {
          throw new IllegalStateException(
              String.format(Locale.ROOT, "Too many matching keys: more than %d", maxSize));
        }
        this.cut = map.lastKey();
        map.remove(cut);
      }
    }

    public Map<K, V> get() {
      return ImmutableSortedMap.copyOfSorted(map);
    }
  }

  static Expression joinFilters(List<Expression> expressions) {
    Expression result = Expressions.alwaysTrue();
    for (Expression expression : expressions) {
      result = Expressions.and(result, expression);
    }
    return result;

View on GitHub (pinned to 86d9c8fc54)