apache/hadoop · error · UnsupportedOperationException

remove not implemented

Error message

remove not implemented

What it means

ValueIterator.remove() unconditionally throws UnsupportedOperationException. The reduce input is a streamed view over shuffle output and is read-only by design; there is nothing to remove mid-iteration.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/ReduceContextImpl.java:251

      // can't advance it here.
      if (!nextKeyIsSame) {
        throw new NoSuchElementException("iterate past last value");
      }
      // otherwise, go to the next key/value pair
      try {
        nextKeyValue();
        return value;
      } catch (IOException ie) {
        throw new RuntimeException("next value iterator failed", ie);
      } catch (InterruptedException ie) {
        // this is bad, but we can't modify the exception list of java.util
        throw new RuntimeException("next value iterator interrupted", ie);        
      }
    }

    @Override
    public void remove() {
      throw new UnsupportedOperationException("remove not implemented");
    }

    @Override
    public void mark() throws IOException {
      if (getBackupStore() == null) {
        backupStore = new BackupStore<KEYIN,VALUEIN>(conf, taskid);
      }
      isMarked = true;
      if (!inReset) {
        backupStore.reinitialize();
        if (currentKeyLength == -1) {
          // The user has not called next() for this iterator yet, so
          // there is no current record to mark and copy to backup store.
          return;
        }
        assert (currentValueLength != -1);
        int requestedSize = currentKeyLength + currentValueLength + 
          WritableUtils.getVIntSize(currentKeyLength) +

View on GitHub (pinned to 2add963021)

Solutions

1. Delete the remove() call and filter values inside the loop instead
2. When handing the iterator to an API that may call remove(), wrap it with an adapter whose remove() is a no-op
3. Model exclusion by collecting keys/offsets to skip, not by mutating the stream

Example fix

// before
Iterator<VALUEIN> it = context.getValues().iterator();
it.next();
it.remove();
// after: filter instead of remove
while (it.hasNext()) {
  VALUEIN v = it.next();
  if (!skip(v)) { context.write(key, v); }
}

if (iteratorHasRemove(it)) { /* refuse to adapt remove() */ }

static Iterator<VALUEIN> readOnly(final Iterator<VALUEIN> it) {
  return new Iterator<VALUEIN>() {
    @Override public boolean hasNext() { return it.hasNext(); }
    @Override public VALUEIN next() { return it.next(); }
    @Override public void remove() { /* no-op: reduce stream is read-only */ }
  };
}

try {
  it.remove();
} catch (UnsupportedOperationException e) {
  log.warn("Values iterator does not support remove; filtering instead");
}

When it happens

Trigger: Reducer code calls context.getValues().iterator().remove(); a library adapting the values iterator to another API invokes remove() as part of its contract.

Common situations: Porting collection-based code to a reducer; using collection helpers (Guava, Streams) that call remove() when filtering; wrapping the iterator in a class expecting a mutable ListIterator.

Related errors

• iterate past last value
• hasNext failed
• next value iterator failed
• Reset called without a previous mark
• No more corrupt file blocks

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/213694459d2ee175. Report an issue: GitHub.