What causes "not implemented" in apache/hadoop?

Delete the remove() call — collect surviving values into your own list if you need to prune.

apache/hadoop · error · RuntimeException

not implemented

Error message

not implemented

What it means

The value iterator handed to the old-API reduce() does not implement remove(); calling it throws this RuntimeException ('not implemented'). The iterator is a live cursor over the sorted merge stream, so element removal has no meaningful semantics. This mirrors the standard JDK practice of unsupported optional Iterator operations, but with RuntimeException rather than UnsupportedOperationException.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java:1637

    public boolean hasNext() { return hasNext; }

    private int ctr = 0;
    public VALUE next() {
      if (!hasNext) {
        throw new NoSuchElementException("iterate past last value");
      }
      try {
        readNextValue();
        readNextKey();
      } catch (IOException ie) {
        throw new RuntimeException("problem advancing post rec#"+ctr, ie);
      }
      reporter.progress();
      return value;
    }

    public void remove() { throw new RuntimeException("not implemented"); }

    /// Auxiliary methods

    /** Start processing next unique key. */
    public void nextKey() throws IOException {
      // read until we find a new key
      while (hasNext) { 
        readNextKey();
      }
      ++ctr;
      
      // move the next key to the current one
      KEY tmpKey = key;
      key = nextKey;
      nextKey = tmpKey;
      hasNext = more;
    }

View on GitHub (pinned to 2add963021)

Solutions

Delete the remove() call — collect surviving values into your own list if you need to prune.
Wrap the iterator in a read-only adapter before passing it to third-party code that might mutate.

Example fix

// before
Iterator<VALUE> it = values.iterator();
it.next();
it.remove();

// after
List<VALUE> kept = new ArrayList<>();
for (VALUE v : values) { if (keep(v)) kept.add(v); }

Defensive patterns

Strategy: validation

Validate before calling

// the values iterator is read-only: filter into your own collection instead
List<VALUE> kept = new ArrayList<>();
for (VALUE v : values) { if (keep(v)) kept.add(v); }

Prevention

Never call remove() on framework-supplied iterators.
Adapt to a List first when handing values to mutation-prone utilities.

When it happens

Trigger: User reduce code or a collection-adapting utility calls remove() on the values iterator inside reduce(); generic algorithms (filter/transform helpers) written against java.util.Iterator that prune as they go.

Common situations: Porting in-memory reduce-side aggregation code that mutated the source list; passing the iterator into libraries that may call remove().

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/25db9c3e7771d515. Report an issue: GitHub.