apache/iceberg · error · IllegalArgumentException

Input malformed or exceeded maximum multipart upload size 5G

Error message

Input malformed or exceeded maximum multipart upload size 5GB: %s

What it means

IllegalArgumentException thrown from the S3FileIOProperties constructor when the s3.multipart-size property cannot be parsed as an int (NumberFormatException from PropertyUtil.propertyAsInt). The multipart size must be a valid integer within the S3 multipart upload limits (up to 5GB per part); malformed or out-of-range values are rejected at S3FileIO construction time with this message.

Source

Thrown at aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java:635

            properties, MULTIPART_UPLOAD_THREADS, Runtime.getRuntime().availableProcessors());
    this.isPathStyleAccess =
        PropertyUtil.propertyAsBoolean(properties, PATH_STYLE_ACCESS, PATH_STYLE_ACCESS_DEFAULT);
    this.isUseArnRegionEnabled =
        PropertyUtil.propertyAsBoolean(
            properties, USE_ARN_REGION_ENABLED, USE_ARN_REGION_ENABLED_DEFAULT);
    this.isAccelerationEnabled =
        PropertyUtil.propertyAsBoolean(
            properties, ACCELERATION_ENABLED, ACCELERATION_ENABLED_DEFAULT);
    this.isDualStackEnabled =
        PropertyUtil.propertyAsBoolean(properties, DUALSTACK_ENABLED, DUALSTACK_ENABLED_DEFAULT);
    this.isCrossRegionAccessEnabled =
        PropertyUtil.propertyAsBoolean(
            properties, CROSS_REGION_ACCESS_ENABLED, CROSS_REGION_ACCESS_ENABLED_DEFAULT);
    try {
      this.multiPartSize =
          PropertyUtil.propertyAsInt(properties, MULTIPART_SIZE, MULTIPART_SIZE_DEFAULT);
    } catch (NumberFormatException e) {
      throw new IllegalArgumentException(
          String.format(
              "Input malformed or exceeded maximum multipart upload size 5GB: %s",
              properties.get(MULTIPART_SIZE)));
    }
    this.multipartThresholdFactor =
        PropertyUtil.propertyAsDouble(
            properties, MULTIPART_THRESHOLD_FACTOR, MULTIPART_THRESHOLD_FACTOR_DEFAULT);
    Preconditions.checkArgument(
        multipartThresholdFactor >= 1.0, "Multipart threshold factor must be >= to 1.0");
    Preconditions.checkArgument(
        multiPartSize >= MULTIPART_SIZE_MIN,
        "Minimum multipart upload object size must be larger than 5 MB.");
    this.stagingDirectory =
        PropertyUtil.propertyAsString(
            properties, STAGING_DIRECTORY, System.getProperty("java.io.tmpdir"));
    String aclType = properties.get(ACL);
    this.acl = ObjectCannedACL.fromValue(aclType);
    Preconditions.checkArgument(

View on GitHub (pinned to 86d9c8fc54)

Solutions

Set s3.multipart-size to a plain integer number of bytes within int range and >= 5MB (e.g. 104857600 for 100MB).
Remove any unit suffixes or formatting — the property does not accept '5GB' or '512MB' strings.
Verify the property key is s3.multipart-size and it isn't inheriting a bad value from an outer configuration (Spark conf, catalog properties).
If you need larger parts, note the S3 limit is 5GB per part — values above that are invalid regardless.

Example fix

// before
Map<String, String> props = Map.of("s3.multipart-size", "10GB"); // NumberFormatException -> IllegalArgumentException
// after
Map<String, String> props = Map.of("s3.multipart-size", String.valueOf(512 * 1024 * 1024)); // "536870912"

Defensive patterns

Strategy: validation

Validate before calling

String multipartSize = properties.get("s3.multipart-size");
if (multipartSize != null) {
  long v = Long.parseLong(multipartSize.trim()); // validate plain integer
  if (v < 5 * 1024 * 1024 || v > Integer.MAX_VALUE) {
    throw new IllegalArgumentException("s3.multipart-size must be 5MB..2GB integer bytes");
  }
}

Prevention

Always specify s3.multipart-size as a plain integer byte count, no unit suffixes
Keep values >= 5MB (S3 minimum part size) and within int range
Check catalog/Spark conf layers for inherited bad values
Add a config sanity check at job startup before creating S3FileIO

When it happens

Trigger: Constructing S3FileIO/S3FileIOProperties with properties containing s3.multipart-size set to a non-numeric string or a value exceeding Integer.MAX_VALUE (e.g. '6GB' as a plain string, or 6000000000).

Common situations: Setting s3.multipart-size=10GB and expecting byte-string parsing (only plain integers are supported); typos like '512 MB'; copying a Hadoop fs.s3a multipart setting name/value; value exceeding the 5GB S3 per-part limit expressed as an int-unparseable number.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/d12f8ca10acb188d. Report an issue: GitHub.