apache/hadoop · error · IllegalArgumentException

Invalid outputs length

Error message

Invalid outputs length

What it means

The HHXOR (Hitchhiker-XOR) decoding step requires outputs.length == erasedIndexes.length * subPacketSize (subPacketSize 2): every erased unit being recovered needs one output buffer per sub-stripe. An RS-style output array with one buffer per erased unit fails this check with IllegalArgumentException.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/HHXORErasureDecodingStep.java:94

    performCoding(inputBuffers, outputBuffers);
  }

  private void performCoding(ByteBuffer[] inputs, ByteBuffer[] outputs)
      throws IOException {
    final int numDataUnits = rsRawDecoder.getNumDataUnits();
    final int numParityUnits = rsRawDecoder.getNumParityUnits();
    final int numTotalUnits = numDataUnits + numParityUnits;
    final int subPacketSize = getSubPacketSize();

    ByteBuffer fisrtValidInput = HHUtil.findFirstValidInput(inputs);
    final int bufSize = fisrtValidInput.remaining();

    if (inputs.length != numTotalUnits * getSubPacketSize()) {
      throw new IllegalArgumentException("Invalid inputs length");
    }

    if (outputs.length != erasedIndexes.length * getSubPacketSize()) {
      throw new IllegalArgumentException("Invalid outputs length");
    }

    // notes:inputs length = numDataUnits * subPacketizationSize
    // first numDataUnits length is first sub-stripe,
    // second numDataUnits length is second sub-stripe
    ByteBuffer[][] newIn = new ByteBuffer[subPacketSize][numTotalUnits];
    for (int i = 0; i < subPacketSize; ++i) {
      for (int j = 0; j < numTotalUnits; ++j) {
        newIn[i][j] = inputs[i * numTotalUnits + j];
      }
    }

    ByteBuffer[][] newOut = new ByteBuffer[subPacketSize][erasedIndexes.length];
    for (int i = 0; i < subPacketSize; ++i) {
      for (int j = 0; j < erasedIndexes.length; ++j) {
        newOut[i][j] = outputs[i * erasedIndexes.length + j];
      }
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Size outputs as erasedIndexes.length * 2 buffers (subPacketSize 2)
2. Keep erasedIndexes consistent with the output blocks handed to the step constructor
3. Use HHXORErasureCodec's decoder so buffer shape matches the codec's expectations

Example fix

// before
ByteBuffer[] outputs = new ByteBuffer[erasedIndexes.length]; // one per erased unit
// throws: Invalid outputs length

// after
ByteBuffer[] outputs = new ByteBuffer[erasedIndexes.length * 2]; // subPacketSize 2

int sub = 2;
if (outputBuffers.length != erasedIndexes.length * sub) {
  throw new IllegalStateException("Allocate " + erasedIndexes.length * sub
      + " output buffers (" + erasedIndexes.length + " erased units x " + sub + ")");
}

try {
  step.performCoding(in, out);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("outputs length")) reallocateOutputsAndRetry();
}

When it happens

Trigger: Calling HHXORErasureDecodingStep.performCoding with outputs sized erasedIndexes.length instead of erasedIndexes.length * 2, or an erasedIndexes array that disagrees with the number of output blocks passed in.

Common situations: Reusing RS decode scaffolding under a hhxor policy without doubling output allocation; inconsistent erasedIndexes vs outputBlocks when the step is constructed manually.

Related errors

• Invalid inputs length
• Invalid inputs length
• Invalid outputs length
• FileSystem ${item.fs.getUri()} does not support Erasure Codi
• Codec not configured for custom codec {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e93bf9811be87e35. Report an issue: GitHub.