apache/hadoop · error · RuntimeException

hasNext failed

Error message

hasNext failed

What it means

ReduceContextImpl.ValueIterator.hasNext(), when replaying marked values (inReset after mark()/reset()), delegates to BackupStore.hasNext(); any exception there is wrapped as RuntimeException('hasNext failed', e) after printStackTrace. The underlying failure — typically local-disk IO while reading the backup store's spill files — is in the cause chain.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/ReduceContextImpl.java:197

  
  BackupStore<KEYIN,VALUEIN> getBackupStore() {
    return backupStore;
  }
  
  protected class ValueIterator implements ReduceContext.ValueIterator<VALUEIN> {

    private boolean inReset = false;
    private boolean clearMarkFlag = false;

    @Override
    public boolean hasNext() {
      try {
        if (inReset && backupStore.hasNext()) {
          return true;
        } 
      } catch (Exception e) {
        e.printStackTrace();
        throw new RuntimeException("hasNext failed", e);
      }
      return firstValue || nextKeyIsSame;
    }

    @Override
    public VALUEIN next() {
      if (inReset) {
        try {
          if (backupStore.hasNext()) {
            backupStore.next();
            DataInputBuffer next = backupStore.nextValue();
            buffer.reset(next.getData(), next.getPosition(), next.getLength()
                - next.getPosition());
            value = valueDeserializer.deserialize(value);
            return value;
          } else {
            inReset = false;
            backupStore.exitResetMode();

View on GitHub (pinned to 2add963021)

Solutions

1. Inspect the wrapped cause in the task log to identify the real IO failure
2. Check NodeManager local directories for space and disk health; the framework retries the task on another node
3. Avoid mark()/reset() rewind for very large key groups — structure the algorithm as a single pass or use secondary sort

try {
  while (values.hasNext()) { consume(values.next()); }
} catch (RuntimeException e) {
  log.error("Value iteration failed (backup store): {}", e.getCause());
  throw e; // let the task fail and be retried on a healthy node
}

When it happens

Trigger: A reducer uses mark()/reset() to re-iterate a large value group, and during reset-mode iteration the BackupStore spill files under the task's local dirs cannot be read (disk full, failed disk, spill corruption).

Common situations: Reducers that rewind over huge value groups (per-key aggregation needing two passes) on nodes with tight NodeManager local-disk space; disks failing under mapreduce.cluster.local.dir.

Related errors

• next value iterator failed
• iterate past last value
• remove not implemented
• Reset called without a previous mark
• remove not supported.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6df88115e93a25ba. Report an issue: GitHub.