What causes "Insufficient bytes in buffer: ${b.length} - ${off} < ${len}" in apache/iceberg?

Fix off/len so that off + len <= b.length

apache/iceberg · error · IOException

Insufficient bytes in buffer

Error message

Insufficient bytes in buffer: ${b.length} - ${off} < ${len}

What it means

write(byte[], int off, int len) validates that the buffer actually holds len bytes starting at off (b.length - off >= len). If not, the requested range would run past the end of the array, so it throws IOException before touching the cipher.

Solutions

Fix off/len so that off + len <= b.length
Pass the buffer's actual used length, not its capacity or another buffer's length
Validate the range in the caller before invoking write

Example fix

// before
out.write(buffer, 0, otherBuffer.length);
// after
out.write(buffer, 0, buffer.length);

Defensive patterns

Strategy: validation

Validate before calling

static void checkRange(byte[] b, int off, int len) {
  if (off < 0 || len < 0 || b.length - off < len) {
    throw new IllegalArgumentException("off+len exceeds buffer: " + off + "+" + len + ">" + b.length);
  }
}

Try / catch

try {
  out.write(buf, off, len);
} catch (IOException e) {
  if (e.getMessage().startsWith("Insufficient bytes in buffer")) {
    throw new IllegalArgumentException("bad off/len for buffer of size " + buf.length, e);
  } else { throw e; }
}

Prevention

Always derive len from the same buffer you pass
Never pass another buffer's length/capacity as len
Validate off + len <= b.length at call sites that compute slices dynamically

When it happens

Trigger: Calling write(b, off, len) where off + len > b.length, e.g. passing a wrong off, a len taken from another buffer's length, or a slice length computed incorrectly.

Common situations: Wrapping AesGcmOutputStream in code that copies buffer-handling arithmetic from a different stream; off-by-one errors when encrypting a sub-range; passing the capacity rather than the filled length of a reusable buffer.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/3235cf27bcfadef4. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/encryption/AesGcmOutputStream.java:82

  @Override
  public void write(int b) throws IOException {
    singleByte[0] = (byte) (b & 0x000000FF);
    write(singleByte);
  }

  @Override
  public void write(byte[] b, int off, int len) throws IOException {
    if (isClosed) {
      throw new IOException("Writing to closed stream");
    }

    if (!isHeaderWritten) {
      writeHeader();
    }

    if (b.length - off < len) {
      throw new IOException(
          "Insufficient bytes in buffer: " + b.length + " - " + off + " < " + len);
    }

    int remaining = len;
    int offset = off;

    while (remaining > 0) {
      int freeBlockBytes = plainBlock.length - positionInPlainBlock;
      int toWrite = Math.min(freeBlockBytes, remaining);

      System.arraycopy(b, offset, plainBlock, positionInPlainBlock, toWrite);
      positionInPlainBlock += toWrite;
      offset += toWrite;
      remaining -= toWrite;

      if (positionInPlainBlock == plainBlock.length) {
        encryptAndWriteBlock();
      }

View on GitHub (pinned to 86d9c8fc54)