{"record":{"id":"b5cd7fbd368a79be","repo":"apache/hadoop","slug":"invalid-buffer-not-of-length-b5cd7f","errorCode":null,"errorMessage":"Invalid buffer not of length {}","messagePattern":"Invalid buffer not of length (.+?)","errorType":"exception","errorClass":"HadoopIllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/ByteArrayDecodingState.java","lineNumber":129,"sourceCode":"    if (validInputs < decoder.getNumDataUnits()) {\n      throw new HadoopIllegalArgumentException(\n          \"No enough valid inputs are provided, not recoverable\");\n    }\n  }\n\n  /**\n   * Check and ensure the buffers are of the desired length.\n   * @param buffers the buffers to check\n   */\n  void checkOutputBuffers(byte[][] buffers) {\n    for (byte[] buffer : buffers) {\n      if (buffer == null) {\n        throw new HadoopIllegalArgumentException(\n            \"Invalid buffer found, not allowing null\");\n      }\n\n      if (buffer.length != decodeLength) {\n        throw new HadoopIllegalArgumentException(\n            \"Invalid buffer not of length \" + decodeLength);\n      }\n    }\n  }\n}\n","sourceCodeStart":111,"sourceCodeEnd":135,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/ByteArrayDecodingState.java#L111-L135","documentation":"During byte-array decode setup, every output buffer must match the decode length derived from the first valid input (buffer.length != decodeLength -> HadoopIllegalArgumentException). Output arrays are where recovered data is written, so each must be exactly the stripe length the decoder works with.","triggerScenarios":"Calling decode with output byte[] arrays allocated at a different size than the inputs — e.g., outputs sized from an older cellSize after a policy change, or allocated before the input length was known.","commonSituations":"Policy changes (cell size / stripe size) applied to inputs but not output allocation; outputs allocated with capacity instead of exact length; mixed buffer caches keyed by wrong size.","solutions":["Allocate each output as new byte[inputLen] where inputLen is the common non-null input length","Recompute output allocation whenever the schema or cell size changes instead of caching sizes","Pre-validate outputs against the first valid input's length before calling decode"],"exampleFix":"// before\nbyte[][] outputs = { new byte[oldCellSize] }; // != decodeLength\ndecoder.decode(inputs, erased, outputs); // throws\n\n// after\nint len = CoderUtil.findFirstValidInput(inputs).length;\nbyte[][] outputs = new byte[erasedIndexes.length][len];","handlingStrategy":"validation","validationCode":"int len = CoderUtil.findFirstValidInput(inputs).length;\nfor (byte[] out : outputs) {\n  if (out == null || out.length != len) {\n    throw new IllegalStateException(\"Each output must be byte[\" + len + \"]\");\n  }\n}\ndecoder.decode(inputs, erased, outputs);","typeGuard":null,"tryCatchPattern":"try {\n  decoder.decode(inputs, erased, outputs);\n} catch (HadoopIllegalArgumentException e) {\n  if (e.getMessage().startsWith(\"Invalid buffer not of length\")) {\n    outputs = allocateOutputs(len); decoder.decode(inputs, erased, outputs);\n  }\n}","preventionTips":["Derive output size from the actual inputs at call time, never from cached config constants","Reallocate decode buffers whenever EC policy cell/stripe size changes","Pre-validate output lengths together with null checks in one guard"],"tags":["erasure-coding","buffer-validation","hadoop-common"],"backgroundTag":"buffer-length-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}