Excess fractional digits are discarded toward zero, matching ClickHouse's own behavior and\n * the truncation {@link BinaryStreamUtils#writeDecimal} would otherwise apply. The result is then\n * bounded by the declared precision: {@code writeDecimal} only range-checks against the backing\n * storage width (32/64/128/256 bits, selected from the precision), which is wider than the\n * declared type, and ClickHouse's RowBinary reader does not re-check. Without this, a value such\n * as {@code 100000} would be stored in a {@code Decimal(5, 0)} column whose declared maximum is\n * {@code 99999}.\n */\n static BigDecimal truncateAndCheckDecimal(BigDecimal value, int precision, int scale) {\n BigDecimal truncated = value.setScale(scale, RoundingMode.DOWN);\n if (truncated.unscaledValue().abs().compareTo(DECIMAL_BOUNDS[precision]) >= 0) {\n throw new IllegalArgumentException(\n \"value \" + value + \" is out of range for Decimal(\" + precision + \", \" + scale + \")\");\n }\n return truncated;\n }\n\n /**\n * Encodes a timestamp into ClickHouse's {@code DateTime64(precision)} representation: a signed\n * 64-bit integer counting ticks of size 10-precision seconds since the Unix epoch.\n *\n *
Accepts either a Joda {@link ReadableInstant} (millisecond precision) or a {@link\n * java.time.Instant} (nanosecond precision). Sub-tick fractions are truncated toward negative\n * infinity, matching ClickHouse's own encoding for negative timestamps.\n */\n static long encodeDateTime64(Object value, int precision) {\n long epochSecond;\n int nanoOfSecond;\n if (value instanceof java.time.Instant) {\n java.time.Instant inst = (java.time.Instant) value;","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/clickhouse/src/main/java/org/apache/beam/sdk/io/clickhouse/ClickHouseWriter.java#L57-L93","documentation":"Thrown by ClickHouseWriter.truncateAndCheckDecimal when a BigDecimal, after truncation to the column scale, still exceeds the maximum magnitude of a ClickHouse Decimal of the given precision (unscaled abs value >= 10^precision). The writer rejects values that would not fit rather than silently corrupting data.","triggerScenarios":"Calling writeValue/decimalValue with a BigDecimal whose integer part has more digits than the target Decimal precision allows — e.g. 100000 into Decimal(5,0) (max 99999), or 123.456 into Decimal(4,2).","commonSituations":"Aggregations (SUM, multiplication) producing values larger than the declared column precision; mismatch between the Beam Row schema and the actual ClickHouse table; ingesting source data whose scale/precision assumptions changed.","solutions":["Pre-truncate or rescale the value before writing so it fits the declared precision/scale.","Alter the ClickHouse column to a wider type (Decimal64/Decimal128) if larger values are legitimate.","Validate upstream data ranges against the target precision before the write.","Align the Beam schema's Decimal precision/scale with the ClickHouse table definition."],"exampleFix":"// before\nwriter.decimalValue(new BigDecimal(\"100000\")); // Decimal(5,0) max is 99999\n// after\nBigDecimal v = new BigDecimal(\"100000\").setScale(0, RoundingMode.DOWN);\nif (v.abs().compareTo(new BigDecimal(\"99999\")) > 0) { throw new ValidationException(\"too large\"); }\nwriter.decimalValue(v);","handlingStrategy":"validation","validationCode":"import java.math.*;\nboolean fitsClickHouseDecimal(BigDecimal value, int precision, int scale) {\n BigDecimal truncated = value.setScale(scale, RoundingMode.DOWN);\n BigDecimal max = BigDecimal.TEN.pow(precision).subtract(BigDecimal.ONE);\n return truncated.abs().compareTo(max) <= 0;\n}","typeGuard":"boolean isBigDecimal(Object v) { return v instanceof java.math.BigDecimal; }","tryCatchPattern":"try {\n writer.decimalValue(bigDecimalValue);\n} catch (IllegalArgumentException e) {\n if (e.getMessage() != null && e.getMessage().contains(\"out of range for Decimal\")) {\n throw new DataException(\"Value exceeds ClickHouse Decimal column capacity: \" + e.getMessage());\n }\n throw e;\n}","preventionTips":["Keep Beam Row Decimal schema precision/scale identical to the ClickHouse table DDL.","Range-check aggregated values (SUM/MUL) against column bounds before writing.","Prefer wider types (Decimal(76, s) or Float64) for computed columns.","Truncate with the same RoundingMode.DOWN semantics the writer uses, in preprocessing."],"tags":["clickhouse","decimal","data-validation","beam"],"backgroundTag":"value-out-of-range","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}