What causes "retainAll is not supported." in apache/hadoop?

Implement the intersection manually: iterate with the iterator and it.remove() everything not in the keep-collection (iterator remove is supported in LightWeightHashSet).

apache/hadoop · error · UnsupportedOperationException

retainAll is not supported.

Error message

retainAll is not supported.

What it means

retainAll is an optional java.util.Set operation that LightWeightHashSet intentionally does not implement - it throws UnsupportedOperationException unconditionally. add/remove/removeAll/contains are all implemented, so code that assumes the full Set interface works until it performs an intersection.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LightWeightHashSet.java:648

        return false;
      }
    }
    return true;
  }

  @Override
  public boolean removeAll(Collection<?> c) {
    boolean changed = false;
    Iterator<?> iter = c.iterator();
    while (iter.hasNext()) {
      changed |= remove(iter.next());
    }
    return changed;
  }

  @Override
  public boolean retainAll(Collection<?> c) {
    throw new UnsupportedOperationException("retainAll is not supported.");
  }
}

View on GitHub (pinned to 2add963021)

Solutions

Implement the intersection manually: iterate with the iterator and it.remove() everything not in the keep-collection (iterator remove is supported in LightWeightHashSet).
Or build the result without mutating: set.stream().filter(keep::contains).collect(Collectors.toSet()).
Wrap in new HashSet<>(set) when full Set semantics including retainAll are required.

Example fix

// before
set.retainAll(liveIds); // UnsupportedOperationException

// after - iterator-based removal, supported by LightWeightHashSet
for (Iterator<T> it = set.iterator(); it.hasNext(); ) {
  if (!liveIds.contains(it.next())) it.remove();
}

Defensive patterns

Strategy: fallback

Validate before calling

// intersection without retainAll - LightWeightHashSet supports iterator removal
for (Iterator<T> it = set.iterator(); it.hasNext(); ) {
  if (!keep.contains(it.next())) it.remove();
}

Prevention

Treat retainAll as unimplemented on all light-weight Hadoop collections - check instanceof before calling on generic Set references.
Centralize set algebra in helpers that branch on implementation capability.
Unit-test generic collection utilities against these custom Set implementations.

When it happens

Trigger: Any call to set.retainAll(collection) on a LightWeightHashSet or LightWeightLinkedSet instance - including from generic library code (CollectionUtils, set-algebra helpers) that only sees the Set interface.

Common situations: Computing intersections of tracked objects (e.g., 'keep only live replicas'); utilities that call retainAll internally; polymorphic code previously tested only against HashSet.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/05c2b5b8f562cdf9. Report an issue: GitHub.