apache/hadoop · error · ConcurrentModificationException

modification={} != expectedModification = {}

Error message

modification={} != expectedModification = {}

What it means

LightWeightHashSet uses the standard fail-fast iterator pattern: every structural add/remove increments a 'modification' counter, the iterator snapshots it as expectedModification, and next() throws ConcurrentModificationException on mismatch. This fires for single-threaded mutation-during-iteration as well as genuine multi-threaded races - the set itself is not thread-safe.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/LightWeightHashSet.java:552

    private int index = -1;
    /** The next element to return. */
    private LinkedElement<T> next = nextNonemptyEntry();
    private LinkedElement<T> current;

    private LinkedElement<T> nextNonemptyEntry() {
      for (index++; index < entries.length && entries[index] == null; index++);
      return index < entries.length ? entries[index] : null;
    }

    @Override
    public boolean hasNext() {
      return next != null;
    }

    @Override
    public T next() {
      if (modification != expectedModification) {
        throw new ConcurrentModificationException("modification="
            + modification + " != expectedModification = " + expectedModification);
      }
      if (next == null) {
        throw new NoSuchElementException();
      }
      current = next;
      final T e = next.element;
      // find the next element
      final LinkedElement<T> n = next.next;
      next = n != null ? n : nextNonemptyEntry();
      return e;
    }

    @Override
    public void remove() {
      if (current == null) {
        throw new NoSuchElementException();
      }

View on GitHub (pinned to 2add963021)

Solutions

1. Use iterator.remove() for removal-during-iteration - LightWeightHashSet's iterator supports it and resyncs expectedModification (see 3248 for its failure mode).
2. Collect doomed elements into a temp list during iteration, then set.removeAll(doomed) afterwards.
3. Iterate a snapshot: for (T e : new ArrayList<>(set)) - mutations of the original then cannot fail the loop.
4. For real concurrency, synchronize all access externally or switch to ConcurrentHashMap.newKeySet().

Example fix

// before
for (T e : set) {
  if (isStale(e)) set.remove(e); // next() throws ConcurrentModificationException
}

// after - collect-then-remove (no structural change during iteration)
List<T> doomed = new ArrayList<>();
for (T e : set) {
  if (isStale(e)) doomed.add(e);
}
set.removeAll(doomed);

// snapshot before iterating - original may then be mutated freely
Collection<T> snapshot = new ArrayList<>(set);
for (T e : snapshot) { ... }

When it happens

Trigger: for-each or explicit Iterator over the set while calling set.add/remove/clear on it (even in the same thread); or another thread mutating without synchronization while this thread iterates.

Common situations: Cleanup loops doing set.remove(e) inside iteration; listener/callback registries where iteration triggers re-registration; Namenode tables iterated while RPCs mutate them concurrently.

Related errors

• modification={} != startModification = {}
• write has error. bs : pre upload obs[%s] has error.
• flushOrSync has error. bs : pre write obs[%s] has error.
• No key-value to read
• Null element is not supported.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e1eab5993f015507. Report an issue: GitHub.