apache/hadoop · error · IllegalArgumentException

Invalid inputs length

Error message

Invalid inputs length

What it means

HHXORErasureEncodingStep.performCoding validates that the encode input array is sub-packetized: inputs.length must equal numDataUnits * subPacketSize, with subPacketSize 2 for Hitchhiker. Each data unit must be presented as two buffers (the two sub-stripes HH piggybacks on); a flat array of numDataUnits buffers is an IllegalArgumentException.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/coder/HHXORErasureEncodingStep.java:75

  }

  @Override
  public void performCoding(ECChunk[] inputChunks, ECChunk[] outputChunks)
      throws IOException {
    ByteBuffer[] inputBuffers = ECChunk.toBuffers(inputChunks);
    ByteBuffer[] outputBuffers = ECChunk.toBuffers(outputChunks);
    performCoding(inputBuffers, outputBuffers);
  }

  private void performCoding(ByteBuffer[] inputs, ByteBuffer[] outputs)
      throws IOException {
    final int numDataUnits = this.rsRawEncoder.getNumDataUnits();
    final int numParityUnits = this.rsRawEncoder.getNumParityUnits();
    final int subSPacketSize = getSubPacketSize();

    // inputs length = numDataUnits * subPacketSize
    if (inputs.length != numDataUnits * subSPacketSize) {
      throw new IllegalArgumentException("Invalid inputs length");
    }

    if (outputs.length != numParityUnits * subSPacketSize) {
      throw new IllegalArgumentException("Invalid outputs length");
    }

    // first numDataUnits length is first sub-stripe,
    // second numDataUnits length is second sub-stripe
    ByteBuffer[][] hhInputs = new ByteBuffer[subSPacketSize][numDataUnits];
    for (int i = 0; i < subSPacketSize; ++i) {
      for (int j = 0; j < numDataUnits; ++j) {
        hhInputs[i][j] = inputs[i * numDataUnits + j];
      }
    }

    ByteBuffer[][] hhOutputs = new ByteBuffer[subSPacketSize][numParityUnits];
    for (int i = 0; i < subSPacketSize; ++i) {
      for (int j = 0; j < numParityUnits; ++j) {

View on GitHub (pinned to 2add963021)

Solutions

1. Allocate inputs as numDataUnits * 2 buffers (subPacketSize 2), filling both sub-stripes per data unit
2. Confirm outputs are likewise sized numParityUnits * 2 to avoid failing the next check
3. Create the encoder through CodecUtil/HHXORErasureCodec so options and layout stay consistent

Example fix

// before
ByteBuffer[] inputs = new ByteBuffer[numDataUnits]; // flat RS layout
step.performCoding(inputBuffers, outputBuffers); // throws

// after
int subPacket = 2;
ByteBuffer[] inputs = new ByteBuffer[numDataUnits * subPacket];

int sub = 2;
if (inputBuffers.length != encoder.getNumDataUnits() * sub) {
  throw new IllegalArgumentException("HHXOR encode needs "
      + encoder.getNumDataUnits() * sub + " input buffers");
}
encoder or step.performCoding(...);

try {
  step.performCoding(inputs, outputs);
} catch (IllegalArgumentException e) {
  // re-allocate inputs as numDataUnits * 2 and rebuild the step
}

When it happens

Trigger: Running the hhxor encoding step (directly or via HHXORErasureCodec encoder under a hhxor EC policy) with inputs allocated as numDataUnits single buffers, or with arrays sized from options whose unit counts differ from the RS raw encoder inside the step.

Common situations: Code written against the plain RS raw coder reused for hhxor; partial migration of a striping writer to sub-packetization; encoder created with different ErasureCoderOptions than the buffer layout.

Related errors

• Invalid inputs length
• Invalid outputs length
• Invalid outputs length
• FileSystem ${item.fs.getUri()} does not support Erasure Codi
• Codec not configured for custom codec {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b2da41354cdd2d11. Report an issue: GitHub.