{"record":{"id":"504f5f75180204ee","repo":"apache/hadoop","slug":"next-value-iterator-failed","errorCode":null,"errorMessage":"next value iterator failed","messagePattern":"next value iterator failed","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/ReduceContextImpl.java","lineNumber":223,"sourceCode":"        try {\n          if (backupStore.hasNext()) {\n            backupStore.next();\n            DataInputBuffer next = backupStore.nextValue();\n            buffer.reset(next.getData(), next.getPosition(), next.getLength()\n                - next.getPosition());\n            value = valueDeserializer.deserialize(value);\n            return value;\n          } else {\n            inReset = false;\n            backupStore.exitResetMode();\n            if (clearMarkFlag) {\n              clearMarkFlag = false;\n              isMarked = false;\n            }\n          }\n        } catch (IOException e) {\n          e.printStackTrace();\n          throw new RuntimeException(\"next value iterator failed\", e);\n        }\n      } \n\n      // if this is the first record, we don't need to advance\n      if (firstValue) {\n        firstValue = false;\n        return value;\n      }\n      // if this isn't the first record and the next key is different, they\n      // can't advance it here.\n      if (!nextKeyIsSame) {\n        throw new NoSuchElementException(\"iterate past last value\");\n      }\n      // otherwise, go to the next key/value pair\n      try {\n        nextKeyValue();\n        return value;\n      } catch (IOException ie) {","sourceCodeStart":205,"sourceCodeEnd":241,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/ReduceContextImpl.java#L205-L241","documentation":"The same reset-mode path in ValueIterator.next(): while inReset, an IOException from backupStore.next(), nextValue(), or value deserialization while replaying marked values is wrapped as RuntimeException('next value iterator failed', e). It indicates the BackupStore of marked values failed mid-replay, not the normal shuffle input.","triggerScenarios":"Reducer in reset mode after mark()/reset() and BackupStore throws IO reading its spill segments (local disk failure, spill file cleanup); deserialization of a stored value fails because the value Writable readFields raised IO.","commonSituations":"Same as other backup-store failures: two-pass reducers on nodes with unhealthy or full local disks; long-running reducers whose local dir was cleaned under them.","solutions":["Read the cause chain in the task log for the concrete IO error","Verify NodeManager local dirs (space, device health); rely on task retry to move off a bad node","Reduce per-key group size or drop the rewind pattern to avoid BackupStore entirely"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  while (values.hasNext()) { consume(values.next()); }\n} catch (RuntimeException e) {\n  if (\"next value iterator failed\".equals(e.getMessage()) && e.getCause() instanceof IOException) {\n    log.error(\"BackupStore IO during reset-mode replay\", e.getCause());\n  }\n  throw e;\n}","preventionTips":["Keep rewind windows small: mark late, reset early","Ensure value Writable readFields is robust and does not throw on valid data","Watch local dirs health where mark/reset reducers run"],"tags":["mapreduce","reducer","iterator","backup-store","local-disk"],"backgroundTag":"backup-store-read-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}