{"record":{"id":"5ed9bde7d4811a9f","repo":"apache/hadoop","slug":"negative-value-length-not-allowed-valuelength-f","errorCode":null,"errorMessage":"Negative value-length not allowed: {valueLength} for {value}","messagePattern":"Negative value-length not allowed: (.+?) for (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/InMemoryWriter.java","lineNumber":58,"sourceCode":"      new DataOutputStream(new IFileOutputStream(arrayStream));\n  }\n  \n  public void append(K key, V value) throws IOException {\n    throw new UnsupportedOperationException\n    (\"InMemoryWriter.append(K key, V value\");\n  }\n  \n  public void append(DataInputBuffer key, DataInputBuffer value)\n  throws IOException {\n    int keyLength = key.getLength() - key.getPosition();\n    if (keyLength < 0) {\n      throw new IOException(\"Negative key-length not allowed: \" + keyLength + \n                            \" for \" + key);\n    }\n    \n    int valueLength = value.getLength() - value.getPosition();\n    if (valueLength < 0) {\n      throw new IOException(\"Negative value-length not allowed: \" + \n                            valueLength + \" for \" + value);\n    }\n\n    WritableUtils.writeVInt(out, keyLength);\n    WritableUtils.writeVInt(out, valueLength);\n    out.write(key.getData(), key.getPosition(), keyLength); \n    out.write(value.getData(), value.getPosition(), valueLength); \n  }\n\n  public void close() throws IOException {\n    // Write EOF_MARKER for key/value length\n    WritableUtils.writeVInt(out, IFile.EOF_MARKER);\n    WritableUtils.writeVInt(out, IFile.EOF_MARKER);\n    \n    // Close the stream \n    out.close();\n    out = null;\n  }","sourceCodeStart":40,"sourceCodeEnd":76,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/InMemoryWriter.java#L40-L76","documentation":"InMemoryWriter.append(DataInputBuffer, DataInputBuffer) computes valueLength as value.getLength() - value.getPosition(); a negative value means the value buffer's position is past its length, so the record would carry a nonsense length. The append fails fast instead of writing corrupt IFile output.","triggerScenarios":"Calling append with a value DataInputBuffer that was already consumed or never reset; reader loops handing the same buffer to multiple writers after advancing it.","commonSituations":"Buffer-reuse patterns in custom merge/copy code; framework-internal occurrences are rare and version-specific.","solutions":["reset() the value buffer to its data before appending: value.reset(valueBytes, 0, valueBytes.length).","Allocate or rebind fresh buffers per append instead of passing consumed ones.","For framework-internal occurrences, capture the stack trace and report upstream."],"exampleFix":"// before: value buffer position advanced past its data\nwriter.append(keyBuf, consumedValueBuf);\n// after: re-bind before appending\nconsumedValueBuf.reset(valueBytes, 0, valueBytes.length);\nwriter.append(keyBuf, consumedValueBuf);","handlingStrategy":"validation","validationCode":"// Guard the value buffer before append\nif (valueBuf.getLength() - valueBuf.getPosition() < 0) {\n  valueBuf.reset(valueBytes, 0, valueBytes.length);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Re-bind value buffers with reset() after any read that advances their position.","Prefer freshly allocated buffers per record when reuse logic is error-prone."],"tags":["hadoop","mapreduce","shuffle","ifile","buffer","validation"],"backgroundTag":"invalid-buffer-state","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}