apache/hadoop · error · IllegalArgumentException

Invalid inputs length

Error message

Invalid inputs length

What it means

HHXORErasureDecodingStep.performCoding (the Hitchhiker-XOR decode step) validates that the input buffer array is sub-packetized: inputs.length must equal (numDataUnits + numParityUnits) * subPacketSize, where subPacketSize is 2 (HHErasureCodingStep.SUB_PACKET_SIZE). Unlike plain RS decoding, each logical unit contributes two buffers (two sub-stripes), so a flat RS-style array is rejected with IllegalArgumentException.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/HHXORErasureDecodingStep.java:90

    }

    ByteBuffer[] inputBuffers = ECChunk.toBuffers(inputChunks);
    ByteBuffer[] outputBuffers = ECChunk.toBuffers(outputChunks);
    performCoding(inputBuffers, outputBuffers);
  }

  private void performCoding(ByteBuffer[] inputs, ByteBuffer[] outputs)
      throws IOException {
    final int numDataUnits = rsRawDecoder.getNumDataUnits();
    final int numParityUnits = rsRawDecoder.getNumParityUnits();
    final int numTotalUnits = numDataUnits + numParityUnits;
    final int subPacketSize = getSubPacketSize();

    ByteBuffer fisrtValidInput = HHUtil.findFirstValidInput(inputs);
    final int bufSize = fisrtValidInput.remaining();

    if (inputs.length != numTotalUnits * getSubPacketSize()) {
      throw new IllegalArgumentException("Invalid inputs length");
    }

    if (outputs.length != erasedIndexes.length * getSubPacketSize()) {
      throw new IllegalArgumentException("Invalid outputs length");
    }

    // notes:inputs length = numDataUnits * subPacketizationSize
    // first numDataUnits length is first sub-stripe,
    // second numDataUnits length is second sub-stripe
    ByteBuffer[][] newIn = new ByteBuffer[subPacketSize][numTotalUnits];
    for (int i = 0; i < subPacketSize; ++i) {
      for (int j = 0; j < numTotalUnits; ++j) {
        newIn[i][j] = inputs[i * numTotalUnits + j];
      }
    }

    ByteBuffer[][] newOut = new ByteBuffer[subPacketSize][erasedIndexes.length];
    for (int i = 0; i < subPacketSize; ++i) {

View on GitHub (pinned to 2add963021)

Solutions

1. Allocate inputs as (numDataUnits + numParityUnits) * 2 buffers (subPacketSize 2), laid out as consecutive sub-stripes
2. Ensure the ErasureCoderOptions used to create the underlying RS decoder match the counts used to size the arrays
3. Obtain encoding/decoding steps via HHXORErasureCodec.createEncoder()/createDecoder() rather than instantiating the step directly

Example fix

// before
int total = numDataUnits + numParityUnits;
ByteBuffer[] inputs = new ByteBuffer[total]; // RS-style, too short
step.performCoding(inputChunks, outputChunks); // throws

// after
int subPacket = 2;
ByteBuffer[] inputs = new ByteBuffer[total * subPacket]; // one buffer per sub-stripe per unit

int sub = 2; // HHErasureCodingStep SUB_PACKET_SIZE
int expected = (rsDecoder.getNumDataUnits() + rsDecoder.getNumParityUnits()) * sub;
if (inputBuffers.length != expected) {
  throw new IllegalArgumentException("HHXOR decode needs " + expected
      + " inputs (" + (expected / sub) + " units x " + sub + " sub-packets)");
}
step.performCoding(inputChunks, outputChunks);

try {
  step.performCoding(in, out);
} catch (IllegalArgumentException e) {
  // length contract violated; recompute allocation from coder options and retry once
}

When it happens

Trigger: Invoking HHXORErasureDecodingStep.performCoding (directly or through the HHXORErasureCodec decoder on a hhxor EC policy) with inputs sized numTotalUnits instead of numTotalUnits * 2, or with buffers allocated from different ErasureCoderOptions than the RS raw decoder held by the step.

Common situations: Porting stripe-level code written for the RS codec to the hhxor codec without doubling buffer counts; constructing the decoding step by hand instead of obtaining it from HHXORErasureCodec; unit-count mismatches between the coder options and buffer allocation.

Related errors

• Invalid outputs length
• Invalid inputs length
• Invalid outputs length
• FileSystem ${item.fs.getUri()} does not support Erasure Codi
• Codec not configured for custom codec {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/457a8e9d16ab37b3. Report an issue: GitHub.