{"record":{"id":"20bbf86842195bf8","repo":"apache/hadoop","slug":"rec-recno-failed-to-skip-past-value-of-length","errorCode":null,"errorMessage":"Rec# {recNo}: Failed to skip past value of length: {currentValueLength}","messagePattern":"Rec# (.+?): Failed to skip past value of length: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/InMemoryReader.java","lineNumber":124,"sourceCode":"      // Record the byte\n      bytesRead += currentKeyLength;\n      return true;\n    } catch (IOException ioe) {\n      dumpOnError();\n      throw ioe;\n    }\n  }\n  \n  public void nextRawValue(DataInputBuffer value) throws IOException {\n    try {\n      int pos = memDataIn.getPosition();\n      byte[] data = memDataIn.getData();\n      value.reset(data, pos, currentValueLength);\n\n      // Position for the next record\n      long skipped = memDataIn.skip(currentValueLength);\n      if (skipped != currentValueLength) {\n        throw new IOException(\"Rec# \" + recNo + \n            \": Failed to skip past value of length: \" + \n            currentValueLength);\n      }\n      // Record the byte\n      bytesRead += currentValueLength;\n\n      ++recNo;\n    } catch (IOException ioe) {\n      dumpOnError();\n      throw ioe;\n    }\n  }\n    \n  public void close() {\n    // Release\n    dataIn = null;\n    buffer = null;\n      // Inform the MergeManager","sourceCodeStart":106,"sourceCodeEnd":142,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/InMemoryReader.java#L106-L142","documentation":"InMemoryReader.nextRawValue() exposes the value bytes then skips currentValueLength bytes to advance to the next record. A skip returning fewer bytes means the in-memory segment holds less data than the record header declared, i.e. corrupted in-memory map output. dumpOnError() dumps the segment before rethrowing.","triggerScenarios":"Same class as the key-skip failure: corrupt in-memory segment from a bad fetch, wrong reader buffer boundaries, or IFile serialization mismatch, detected one field later while positioning for the next record.","commonSituations":"Shuffle data corruption on a specific map output surfacing during merge; framework-internal path used by MergeManagerImpl, not direct user API.","solutions":["Inspect the dumpOnError() segment dump logged with the failure.","Check the NodeManager that served the failing map output (disk, logs, recent restarts).","Retry so the map is re-fetched or re-run; restart the serving NM if it stays broken.","Open a MapReduce JIRA with the dump and stack trace if reproducible."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch (java.io.IOException e) { if (String.valueOf(e.getMessage()).contains(\"Failed to skip past value\")) { /* corrupt in-memory segment: diagnostics dumped; retry the job */ } else { throw e; } }","preventionTips":["Treat skip failures as data corruption and gather the dumped segment.","Track which map output/host fed the failing segment to isolate bad nodes."],"tags":["hadoop","mapreduce","shuffle","merge","data-corruption","ifile"],"backgroundTag":"truncated-data","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}