apache/hadoop · error · IllegalArgumentException

Invalid outputs length

Error message

Invalid outputs length

What it means

The HHXOR (Hitchhiker-XOR) encoding step requires outputs.length == numParityUnits * subPacketSize (subPacketSize 2): parity is produced per sub-stripe, so each parity unit yields two output buffers. A single-buffer-per-parity-unit array throws IllegalArgumentException before any coding runs.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/HHXORErasureEncodingStep.java:79

      throws IOException {
    ByteBuffer[] inputBuffers = ECChunk.toBuffers(inputChunks);
    ByteBuffer[] outputBuffers = ECChunk.toBuffers(outputChunks);
    performCoding(inputBuffers, outputBuffers);
  }

  private void performCoding(ByteBuffer[] inputs, ByteBuffer[] outputs)
      throws IOException {
    final int numDataUnits = this.rsRawEncoder.getNumDataUnits();
    final int numParityUnits = this.rsRawEncoder.getNumParityUnits();
    final int subSPacketSize = getSubPacketSize();

    // inputs length = numDataUnits * subPacketSize
    if (inputs.length != numDataUnits * subSPacketSize) {
      throw new IllegalArgumentException("Invalid inputs length");
    }

    if (outputs.length != numParityUnits * subSPacketSize) {
      throw new IllegalArgumentException("Invalid outputs length");
    }

    // first numDataUnits length is first sub-stripe,
    // second numDataUnits length is second sub-stripe
    ByteBuffer[][] hhInputs = new ByteBuffer[subSPacketSize][numDataUnits];
    for (int i = 0; i < subSPacketSize; ++i) {
      for (int j = 0; j < numDataUnits; ++j) {
        hhInputs[i][j] = inputs[i * numDataUnits + j];
      }
    }

    ByteBuffer[][] hhOutputs = new ByteBuffer[subSPacketSize][numParityUnits];
    for (int i = 0; i < subSPacketSize; ++i) {
      for (int j = 0; j < numParityUnits; ++j) {
        hhOutputs[i][j] = outputs[i * numParityUnits + j];
      }
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Allocate outputs as numParityUnits * 2 buffers (subPacketSize 2)
2. Update any shared allocation helpers to take the codec's sub-packet size into account
3. Drive encoding via HHXORErasureCodec so the buffer contract is honored end to end

Example fix

// before
ByteBuffer[] outputs = new ByteBuffer[numParityUnits];
// throws: Invalid outputs length

// after
ByteBuffer[] outputs = new ByteBuffer[numParityUnits * 2]; // subPacketSize 2

int sub = 2;
if (outputBuffers.length != encoder.getNumParityUnits() * sub) {
  throw new IllegalArgumentException("HHXOR encode needs "
      + encoder.getNumParityUnits() * sub + " output buffers");
}

try {
  step.performCoding(inputs, outputs);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("outputs length")) resizeOutputs();
}

When it happens

Trigger: Calling the hhxor encoding step's performCoding with outputs sized numParityUnits instead of numParityUnits * 2, or mixing an RS-shaped output allocation into an hhxor pipeline.

Common situations: Shared parity-allocation helpers written for RS reused under a hhxor policy; refactoring an encoder pipeline where only input sizing was updated.

Related errors

• Invalid inputs length
• Invalid outputs length
• Invalid inputs length
• FileSystem ${item.fs.getUri()} does not support Erasure Codi
• Codec not configured for custom codec {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/699fd15529126d72. Report an issue: GitHub.