apache/hadoop · error · IllegalArgumentException

blockSize({}) > 9

Error message

blockSize({}) > 9

What it means

The upper-bound half of CBZip2OutputStream's constructor validation: blockSize must not exceed 9 (MAX_BLOCKSIZE), because the bzip2 format encodes the block size as a single digit ('BZh1'..'BZh9') and the decoder allocates blockSize*100000-byte buffers accordingly. Anything above 9 throws IllegalArgumentException("blockSize(...) > 9") before any output is written.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/CBZip2OutputStream.java:636

  *             if an I/O error occurs in the specified stream.
  * @throws IllegalArgumentException
  *             if {@code (blockSize < 1) || (blockSize > 9)}
  * @throws NullPointerException
  *             if {@code out == null}.
  *
  * @see #MIN_BLOCKSIZE
  * @see #MAX_BLOCKSIZE
  */
  public CBZip2OutputStream(final OutputStream out, final int blockSize)
      throws IOException {
    super();

    if (blockSize < 1) {
      throw new IllegalArgumentException("blockSize(" + blockSize
          + ") < 1");
    }
    if (blockSize > 9) {
      throw new IllegalArgumentException("blockSize(" + blockSize
          + ") > 9");
    }

    this.blockSize100k = blockSize;
    this.out = out;
    init();
  }

  @Override
  public void write(final int b) throws IOException {
    if (this.out != null) {
      write0(b);
    } else {
      throw new IOException("closed");
    }
  }

  private void writeRun() throws IOException {

View on GitHub (pinned to 2add963021)

Solutions

1. Pass the format-level code, not bytes: divide byte sizes by 100000 and clamp to 1..9.
2. Clamp defensively: Math.min(CBZip2OutputStream.MAX_BLOCKSIZE, Math.max(CBZip2OutputStream.MIN_BLOCKSIZE, blockSize)).
3. Use Bzip2Factory/Configuration plumbing ('bzip2.compress.blocksize', default 9) instead of hand-passing numbers.
4. Add a startup assertion/property check that names the bad value so the fix lands in config, not a stack trace at first write.

Example fix

// before
new CBZip2OutputStream(out, 900000); // meant 900k bytes, passed raw

// after
int bs = Math.max(1, Math.min(9, desiredBytes / 100000));
new CBZip2OutputStream(out, bs); // bs == 9

int bs = (desiredBlockBytes + 99999) / 100000; // bytes -> 1..9 code
bs = Math.max(1, Math.min(9, bs));
new CBZip2OutputStream(out, bs);

When it happens

Trigger: new CBZip2OutputStream(out, blockSize) with a value like 10..900000: callers passing the block size in bytes (e.g. 900000) instead of the 1-9 code, or config/multiplier arithmetic (blockSize * 100) exceeding the range.

Common situations: Unit confusion between 'blocksize' codes (1-9) and byte sizes (100k-900k); copy-paste from tools that accept --bzip2-blocksize=900k; properties scaled twice; upgraded code that used to accept raw byte counts.

Related errors

• blockSize({}) < 1
• offs({}) < 0.
• len({}) < 0.
• offs({}) + len({}) > buf.length({}).
• block overrun

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3852fba6b71a3bef. Report an issue: GitHub.