apache/hadoop · error · HadoopIllegalArgumentException

Invalid buffer found, not allowing null

Error message

Invalid buffer found, not allowing null

What it means

Unlike inputs (where null marks an erased unit), decode outputs must all be allocated: ByteArrayDecodingState.checkOutputBuffers throws HadoopIllegalArgumentException("Invalid buffer found, not allowing null") on the first null output byte[]. The decoder needs somewhere to write every recovered unit.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/rawcoder/ByteArrayDecodingState.java:124

      }

      validInputs++;
    }

    if (validInputs < decoder.getNumDataUnits()) {
      throw new HadoopIllegalArgumentException(
          "No enough valid inputs are provided, not recoverable");
    }
  }

  /**
   * Check and ensure the buffers are of the desired length.
   * @param buffers the buffers to check
   */
  void checkOutputBuffers(byte[][] buffers) {
    for (byte[] buffer : buffers) {
      if (buffer == null) {
        throw new HadoopIllegalArgumentException(
            "Invalid buffer found, not allowing null");
      }

      if (buffer.length != decodeLength) {
        throw new HadoopIllegalArgumentException(
            "Invalid buffer not of length " + decodeLength);
      }
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Allocate a non-null byte[decodeLength] for every output slot before decoding
2. Size the outputs array to the number of units being recovered and fill every element
3. Keep null-marking conventions for inputs only; never for outputs

Example fix

// before
byte[][] outputs = new byte[erasedIndexes.length]; // array of nulls
decoder.decode(inputs, erased, outputs); // throws

// after
byte[][] outputs = new byte[erasedIndexes.length][];
Arrays.fill(outputs, new byte[decodeLength]);

for (byte[] out : outputs) {
  if (out == null) throw new IllegalStateException(
      "Null decode output buffer not allowed; allocate all outputs");
}
decoder.decode(inputs, erased, outputs);

try {
  decoder.decode(inputs, erased, outputs);
} catch (HadoopIllegalArgumentException e) {
  if (e.getMessage().contains("not allowing null")) allocateOutputsAndRetry();
}

When it happens

Trigger: Calling decode(byte[][], int[], byte[][]) with any null entry in the outputs array — e.g., reusing an input array containing nulls as the output allocation, or allocating outputs.length fewer buffers than erased units.

Common situations: Output arrays sized by a stale erased-unit count; nulls leaking in from buffer pools that hand out null on exhaustion; copy-paste of input allocation patterns (where nulls are legal) to outputs.

Related errors

• Invalid buffer found, not allowing null
• Invalid inputs are found, all being null
• Invalid buffer, not of length {}
• Invalid buffer not of length {}
• Invalid buffer not of length {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/bb7bea41425a11d4. Report an issue: GitHub.