What causes "Provided size %s is bigger than max size %s " in apache/beam?

Reduce the declared size to at most 8388608 (8 MB), typically SnowflakeBinary.of(8388608) for max-size columns.

apache/beam · error · IllegalArgumentException

Provided size is bigger than max size

Error message

Provided size %s is bigger than max size %s

What it means

SnowflakeBinary.of()/the SnowflakeBinary(long) constructor validates the declared BINARY column size against MAX_SIZE (Snowflake's maximum binary size, 8388608 bytes). Sizes above the maximum cannot be represented by Snowflake's BINARY type, so the constructor throws an IllegalArgumentException.

Solutions

Reduce the declared size to at most 8388608 (8 MB), typically SnowflakeBinary.of(8388608) for max-size columns.
If the data exceeds 8 MB, store it externally (cloud storage) and keep a reference/URL in the table instead.
Check where the size constant comes from and correct the unit/conversion producing the oversized value.

Example fix

// before
SnowflakeBinary.of(16777216)
// after
SnowflakeBinary.of(8388608)

Defensive patterns

Strategy: validation

Validate before calling

if (size > SnowflakeBinary.MAX_SIZE) {
  throw new IllegalArgumentException("size must be <= " + SnowflakeBinary.MAX_SIZE);
}

Type guard

static boolean isValidBinarySize(long size) { return size >= 0 && size <= 8388608L; }

Try / catch

try {
  SnowflakeBinary col = SnowflakeBinary.of(size);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("bigger than max size")) { size = SnowflakeBinary.MAX_SIZE; }
  else throw e;
}

Prevention

Clamp declared BINARY sizes to SnowflakeBinary.MAX_SIZE (8388608) when porting schemas.
Store oversized blobs in cloud storage and reference them by URL instead.
Add schema-definition unit tests that construct every column type used by the pipeline.

When it happens

Trigger: Creating SnowflakeBinary.of(size) or new SnowflakeBinary(size) with size > 8388608 (Snowflake BINARY max), e.g. SnowflakeBinary.of(16777216).

Common situations: Confusing the Snowflake BINARY limit with other engines' larger limits (e.g. 16 MB/64 MB blobs), or multiplying byte sizes incorrectly when defining table schema columns.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2ea3fd896cd85349. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/data/text/SnowflakeBinary.java:44

public class SnowflakeBinary implements SnowflakeDataType, Serializable {

  public static final Long MAX_SIZE = 8388608L;

  private Long size; // bytes

  public SnowflakeBinary() {}

  public static SnowflakeBinary of() {
    return new SnowflakeBinary();
  }

  public static SnowflakeBinary of(long size) {
    return new SnowflakeBinary(size);
  }

  public SnowflakeBinary(long size) {
    if (size > MAX_SIZE) {
      throw new IllegalArgumentException(
          String.format("Provided size %s is bigger than max size %s ", size, MAX_SIZE));
    }
    this.size = size;
  }

  @Override
  public String sql() {
    if (size != null) {
      return String.format("BINARY(%d)", size);
    }
    return "BINARY";
  }

  public Long getSize() {
    return size;
  }

  public void setSize(Long size) {

View on GitHub (pinned to 12126d8942)