{"record":{"id":"3977226b9308ff19","repo":"apache/hadoop","slug":"invalid-buffer-not-of-length-397722","errorCode":null,"errorMessage":"Invalid buffer not of length {}","messagePattern":"Invalid buffer not of length (.+?)","errorType":"exception","errorClass":"HadoopIllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/ByteArrayEncodingState.java","lineNumber":99,"sourceCode":"\n    ByteBufferEncodingState bbeState = new ByteBufferEncodingState(encoder,\n        encodeLength, newInputs, newOutputs);\n    return bbeState;\n  }\n\n  /**\n   * Check and ensure the buffers are of the desired length.\n   * @param buffers the buffers to check\n   */\n  void checkBuffers(byte[][] buffers) {\n    for (byte[] buffer : buffers) {\n      if (buffer == null) {\n        throw new HadoopIllegalArgumentException(\n            \"Invalid buffer found, not allowing null\");\n      }\n\n      if (buffer.length != encodeLength) {\n        throw new HadoopIllegalArgumentException(\n            \"Invalid buffer not of length \" + encodeLength);\n      }\n    }\n  }\n}\n","sourceCodeStart":81,"sourceCodeEnd":105,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/ByteArrayEncodingState.java#L81-L105","documentation":"ByteArrayEncodingState derives encodeLength from the first valid input's array length and requires every input and output byte[] in the encode call to be exactly that length (buffer.length != encodeLength -> HadoopIllegalArgumentException). All units of a stripe must be identically sized for the raw encoder to process them.","triggerScenarios":"Calling encode(byte[][], byte[][]) where any input or output array differs in length from the first input — short unpadded final chunks, or outputs allocated with a stale size.","commonSituations":"Last-stripe data not zero-padded to the stripe length; cell/stripe size changed by policy but buffer allocation code not updated; outputs allocated from a different constant than inputs.","solutions":["Zero-pad the final chunk so all inputs share one length","Allocate all outputs with the same length as the inputs (first input's length)","Validate lengths of both arrays in one pre-encode check"],"exampleFix":"// before\nbyte[][] inputs = { new byte[cellSize], Arrays.copyOf(data, n) }; // n < cellSize\nencoder.encode(inputs, outputs); // throws\n\n// after\nbyte[][] inputs = new byte[numDataUnits][cellSize];\nSystem.arraycopy(data, 0, inputs[1], 0, n); // padded to cellSize","handlingStrategy":"validation","validationCode":"int len = CoderUtil.findFirstValidInput(inputs).length;\nfor (byte[][] arr : new byte[][][]{inputs, outputs}) {\n  for (byte[] b : arr) {\n    if (b != null && b.length != len) {\n      throw new IllegalStateException(\"All encode buffers must be byte[\" + len + \"]\");\n    }\n  }\n}\nencoder.encode(inputs, outputs);","typeGuard":null,"tryCatchPattern":"try {\n  encoder.encode(inputs, outputs);\n} catch (HadoopIllegalArgumentException e) {\n  if (e.getMessage().startsWith(\"Invalid buffer not of length\")) padAndRetry();\n}","preventionTips":["Zero-pad the last chunk so all encode inputs match the first input's length","Allocate outputs with the same length constant as inputs","Add a length-uniformity assertion in striped-write unit tests"],"tags":["erasure-coding","buffer-validation","hadoop-common"],"backgroundTag":"buffer-length-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}