apache/iceberg · error · IOException

Cannot write block: exceeded Integer.MAX_VALUE blocks

Error message

Cannot write block: exceeded Integer.MAX_VALUE blocks

What it means

The GCM stream format numbers blocks with a 32-bit counter (the block index is encoded into the AAD via a little-endian int). When currentBlockIndex reaches Integer.MAX_VALUE no further block can be keyed, so writing the next block throws IOException to prevent silent nonce/AAD reuse, which would be a cryptographic failure.

Solutions

Increase the block size so the total block count stays far below Integer.MAX_VALUE
Split the data across multiple files/streams instead of one stream
Check configuration for an accidentally small encryption block size

Defensive patterns

Strategy: validation

Validate before calling

// before writing a huge stream
long maxBytes = (long) Integer.MAX_VALUE * blockSizeBytes;
if (expectedTotalBytes > maxBytes) {
  throw new IllegalArgumentException("Split data across files; exceeds max GCM blocks");
}

Try / catch

try {
  out.write(chunk);
} catch (IOException e) {
  if (e.getMessage().contains("exceeded Integer.MAX_VALUE blocks")) {
    throw new IllegalStateException("File too large for single GCM stream; reduce block-size or split file", e);
  } else { throw e; }
}

Prevention

Keep encryption block size at its default; don't shrink it for large files
Estimate block count = totalBytes / blockSize before writing very large files
Split multi-terabyte datasets across multiple encrypted files

When it happens

Trigger: Writing more than Integer.MAX_VALUE blocks (~2 billion 4MiB default blocks, far less with tiny block sizes) into a single AesGcmOutputStream before close; flushBlock boundaries hit at index Integer.MAX_VALUE.

Common situations: Very large single encrypted file writes; a misconfigured tiny block size (write.block-size property) causing block count to explode long before the byte limit matters.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/135577670d2a0846. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/encryption/AesGcmOutputStream.java:147

    targetStream.close();
  }

  @Override
  public long storedLength() throws IOException {
    return targetStream.storedLength();
  }

  private void writeHeader() throws IOException {
    targetStream.write(HEADER_BYTES);
    isHeaderWritten = true;
  }

  private void encryptAndWriteBlock() throws IOException {
    Preconditions.checkState(
        !lastBlockWritten, "Cannot encrypt block: a partial block has already been written");

    if (currentBlockIndex == Integer.MAX_VALUE) {
      throw new IOException("Cannot write block: exceeded Integer.MAX_VALUE blocks");
    }

    if (positionInPlainBlock == 0 && currentBlockIndex != 0) {
      return;
    }

    if (positionInPlainBlock != plainBlock.length) {
      // signal that a partial block has been written and must be the last
      this.lastBlockWritten = true;
    }

    byte[] aad = Ciphers.streamBlockAAD(fileAadPrefix, currentBlockIndex);
    int ciphertextLength =
        gcmEncryptor.encrypt(plainBlock, 0, positionInPlainBlock, cipherBlock, 0, aad);
    targetStream.write(cipherBlock, 0, ciphertextLength);
    positionInPlainBlock = 0;
    currentBlockIndex++;
  }

View on GitHub (pinned to 86d9c8fc54)