apache/beam · error · IllegalArgumentException

" + typeName + " scale cannot exceed " + scale + ": " + v.to

Error message

" + typeName + " scale cannot exceed " + scale + ": " + v.toPlainString()

What it means

When serializing a BigDecimal for a BigQuery NUMERIC column, serializeBigDecimal rejects values whose decimal scale (fraction digits) exceeds the allowed NUMERIC_SCALE of 9. This is a data-precondition enforced because BigQuery NUMERIC cannot store more than 9 decimal places.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BeamRowToStorageApiProto.java:443

    }
    FieldDescriptor valueFieldDescriptor =
        Preconditions.checkNotNull(descriptor.findFieldByName("value"));
    @Nullable
    Object value = toProtoValue(valueFieldDescriptor, valueFieldType, entryValue.getValue());
    if (value != null) {
      builder.setField(valueFieldDescriptor, value);
    }
    return builder.build();
  }

  static ByteString serializeBigDecimalToNumeric(BigDecimal o) {
    return serializeBigDecimal(o, NUMERIC_SCALE, MAX_NUMERIC_VALUE, MIN_NUMERIC_VALUE, "Numeric");
  }

  private static ByteString serializeBigDecimal(
      BigDecimal v, int scale, BigDecimal maxValue, BigDecimal minValue, String typeName) {
    if (v.scale() > scale) {
      throw new IllegalArgumentException(
          typeName + " scale cannot exceed " + scale + ": " + v.toPlainString());
    }
    if (v.compareTo(maxValue) > 0 || v.compareTo(minValue) < 0) {
      throw new IllegalArgumentException(typeName + " overflow: " + v.toPlainString());
    }

    byte[] bytes = v.setScale(scale).unscaledValue().toByteArray();
    // NUMERIC/BIGNUMERIC values are serialized as scaled integers in two's complement form in
    // little endian
    // order. BigInteger requires the same encoding but in big endian order, therefore we must
    // reverse the bytes that come from the proto.
    Bytes.reverse(bytes);
    return ByteString.copyFrom(bytes);
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Round/truncate the value before the sink: value.setScale(9, RoundingMode.HALF_UP)
2. Use BIGNUMERIC column type via the corresponding serializer if more scale is needed
3. Validate/sanitize BigDecimal scales in a transform upstream of the BigQueryIO write

Example fix

// before
BigDecimal rate = numerator.divide(denominator);
row.getDecimal("rate")
// after
BigDecimal rate = numerator.divide(denominator, 9, RoundingMode.HALF_UP);

if (value.scale() > 9) { value = value.setScale(9, RoundingMode.HALF_UP); }

When it happens

Trigger: serializeBigDecimalToNumeric receives a BigDecimal with scale > 9, e.g. produced by computations like division or averaging inside the pipeline before writing to BigQuery NUMERIC.

Common situations: Pipelines computing averages or interest rates produce values like 0.1234567891; BigQuery sink then fails. Common when upstream data or calc precision is not rounded/truncated before the sink.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• " + typeName + " overflow: " + v.toPlainString()
• Unsupported format for BigQuery table path: '{linkedResource
• Reserved field name <field.name()> in user schema.
• Unsupported type <elementType.getType()>
• Received null value for non-nullable field " + fieldDescript

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/30fe4f94b6f8b129. Report an issue: GitHub.