apache/hadoop · error · ConcurrentModificationException

modification=" + modification + " != iterModification = " +

Error message

modification=" + modification + " != iterModification = " + iterModification

What it means

The GSet iterator is fail-fast: every structural put()/remove() increments the set's modification counter, and the iterator compares it with the counter captured at creation (ensureNext), throwing ConcurrentModificationException('modification=X != iterModification = Y'). The set itself is not thread-safe, so mutations from another thread trip the same check.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/LightWeightGSet.java:314

  public class SetIterator implements Iterator<E> {
    /** The starting modification for fail-fast. */
    private int iterModification = modification;
    /** The current index of the entry array. */
    private int index = -1;
    private LinkedElement cur = null;
    private LinkedElement next = nextNonemptyEntry();
    private boolean trackModification = true;

    /** Find the next nonempty entry starting at (index + 1). */
    private LinkedElement nextNonemptyEntry() {
      for(index++; index < entries.length && entries[index] == null; index++);
      return index < entries.length? entries[index]: null;
    }

    private void ensureNext() {
      if (trackModification && modification != iterModification) {
        throw new ConcurrentModificationException("modification=" + modification
            + " != iterModification = " + iterModification);
      }
      if (next != null) {
        return;
      }
      if (cur == null) {
        return;
      }
      next = cur.getNext();
      if (next == null) {
        next = nextNonemptyEntry();
      }
    }

    @Override
    public boolean hasNext() {
      ensureNext();
      return next != null;

View on GitHub (pinned to 2add963021)

Solutions

1. Use iterator.remove() for in-loop deletion - it re-syncs iterModification after removing.
2. Two-phase update: collect targets during iteration, apply put/remove after the loop finishes.
3. For deliberate concurrent designs call setTrackModification(false) and add external locking, accepting that iteration may then observe skips or repeats.

Example fix

// before
for (E e : gset.values()) {
  if (isStale(e)) {
    gset.remove(e.getKey()); // ConcurrentModificationException
  }
}

// after
List<K> stale = new ArrayList<>();
for (E e : gset.values()) {
  if (isStale(e)) {
    stale.add(e.getKey());
  }
}
for (K k : stale) {
  gset.remove(k);
}

// two-phase pattern: no structural change during iteration
List<K> toRemove = new ArrayList<>();
for (E e : gset.values()) {
  if (shouldEvict(e)) {
    toRemove.add(e.getKey());
  }
}
toRemove.forEach(gset::remove);

try {
  for (E e : gset.values()) { ... }
} catch (ConcurrentModificationException e) {
  // restart the iteration on a snapshot; do not swallow and continue
}

When it happens

Trigger: Mutating the set with put()/remove() while iterating values() - including from a callback fired inside the loop or from another thread.

Common situations: Expiry or eviction scans that delete entries during iteration; listener hooks mutating the same set mid-iteration; single-threaded maintenance code later moved into a background thread.

Related errors

• There are no more elements
• There is no current element to remove
• No more items in iterator.
• Interrupted while waiting for IO on channel {}. Total timeou
• key == null

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4363da653684f9f2. Report an issue: GitHub.