{"record":{"id":"1cf0d76308f65391","repo":"apache/hadoop","slug":"invalid-buffer-not-of-length-1cf0d7","errorCode":null,"errorMessage":"Invalid buffer, not of length {}","messagePattern":"Invalid buffer, not of length (.+?)","errorType":"exception","errorClass":"HadoopIllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/ByteBufferDecodingState.java","lineNumber":107,"sourceCode":"        inputOffsets, newOutputs, outputOffsets);\n    return baeState;\n  }\n\n  /**\n   * Check and ensure the buffers are of the desired length and type, direct\n   * buffers or not.\n   * @param buffers the buffers to check\n   */\n  void checkInputBuffers(ByteBuffer[] buffers) {\n    int validInputs = 0;\n\n    for (ByteBuffer buffer : buffers) {\n      if (buffer == null) {\n        continue;\n      }\n\n      if (buffer.remaining() != decodeLength) {\n        throw new HadoopIllegalArgumentException(\n            \"Invalid buffer, not of length \" + decodeLength);\n      }\n      if (buffer.isDirect() != usingDirectBuffer) {\n        throw new HadoopIllegalArgumentException(\n            \"Invalid buffer, isDirect should be \" + usingDirectBuffer);\n      }\n\n      validInputs++;\n    }\n\n    if (validInputs < decoder.getNumDataUnits()) {\n      throw new HadoopIllegalArgumentException(\n          \"No enough valid inputs are provided, not recoverable\");\n    }\n  }\n\n  /**\n   * Check and ensure the buffers are of the desired length and type, direct","sourceCodeStart":89,"sourceCodeEnd":125,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/ByteBufferDecodingState.java#L89-L125","documentation":"For ByteBuffer decode, ByteBufferDecodingState sets decodeLength from the first non-null input's remaining() and requires every non-null input buffer to have exactly that many remaining bytes (mismatch -> HadoopIllegalArgumentException(\"Invalid buffer, not of length <decodeLength>\")). A following check also enforces uniform direct/heap type, so all decode inputs must be same-length, same-type buffers with consistent positions.","triggerScenarios":"Calling RawErasureDecoder.decode(ByteBuffer[], int[], ByteBuffer[]) with buffers whose remaining() differs — e.g., one buffer sliced or read into further than others, a short final chunk not padded, or mixed allocations from direct and heap pools (which additionally trips the isDirect check).","commonSituations":"Reusing buffers that retain positions/limits from previous operations; striped reads with unpadded final chunks; half-migrated code switching from heap to direct buffers so some inputs differ in type and prepared length.","solutions":["Prepare every input buffer to the same remaining() length: rewind/clear, set position, and pad short data to the stripe length","Use one allocation style (all ByteBuffer.allocateDirect or all allocate) per decode call","Pre-check remaining() and isDirect() of all non-null inputs before invoking decode"],"exampleFix":"// before\nByteBuffer[] inputs = { buf0, buf1 }; // buf1.remaining() != buf0.remaining()\ndecoder.decode(inputs, erased, outputs); // throws\n\n// after\nint len = inputs[0].remaining();\nfor (ByteBuffer b : inputs) {\n  if (b != null) b.limit(b.position() + len); // normalize remaining\n}","handlingStrategy":"validation","validationCode":"ByteBuffer first = CoderUtil.findFirstValidInput(inputs);\nint len = first.remaining();\nboolean direct = first.isDirect();\nfor (ByteBuffer b : inputs) {\n  if (b != null && (b.remaining() != len || b.isDirect() != direct)) {\n    throw new IllegalStateException(\n        \"All decode inputs need remaining()==\" + len + \" and isDirect()==\" + direct);\n  }\n}\ndecoder.decode(inputs, erased, outputs);","typeGuard":"boolean uniformInputs(ByteBuffer[] inputs) {\n  ByteBuffer first = null;\n  for (ByteBuffer b : inputs) if (b != null) { first = b; break; }\n  if (first == null) return false;\n  for (ByteBuffer b : inputs) {\n    if (b != null && (b.remaining() != first.remaining()\n        || b.isDirect() != first.isDirect())) return false;\n  }\n  return true;\n}","tryCatchPattern":"try {\n  decoder.decode(inputs, erased, outputs);\n} catch (HadoopIllegalArgumentException e) {\n  if (e.getMessage().contains(\"not of length\")\n      || e.getMessage().contains(\"isDirect\")) {\n    normalizeBuffers(); // set consistent limit/position, one allocation type, retry\n  }\n}","preventionTips":["Reset buffer positions/limits before each decode; don't reuse buffers with stale state","Pick one buffer style (direct vs heap) per code path and stick to it","Pad short final stripes so every buffer's remaining() matches the stripe length"],"tags":["erasure-coding","buffer-validation","nio","hadoop-common"],"backgroundTag":"buffer-length-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}