{"record":{"id":"044ee54c87a3b7bf","repo":"risingwavelabs/risingwave","slug":"support-negative-scale-for-arrow-decimal","errorCode":null,"errorMessage":"support negative scale for arrow decimal","messagePattern":"support negative scale for arrow decimal","errorType":"error_code","errorClass":"ArrayError","httpStatus":null,"severity":"error","filePath":"src/common/src/array/arrow/arrow_impl.rs","lineNumber":1583,"sourceCode":"\nimpl From<&DecimalArray> for arrow_array::StringArray {\n    fn from(array: &DecimalArray) -> Self {\n        let mut builder =\n            arrow_array::builder::StringBuilder::with_capacity(array.len(), array.len() * 8);\n        for value in array.iter() {\n            builder.append_option(value.map(|d| d.to_string()));\n        }\n        builder.finish()\n    }\n}\n\n// This arrow decimal type is used by iceberg source to read iceberg decimal into RW decimal.\nimpl TryFrom<&arrow_array::Decimal128Array> for DecimalArray {\n    type Error = ArrayError;\n\n    fn try_from(array: &arrow_array::Decimal128Array) -> Result<Self, Self::Error> {\n        if array.scale() < 0 {\n            bail!(\"support negative scale for arrow decimal\")\n        }\n\n        // Calculate the max value based on the Arrow decimal's precision\n        // When writing Inf to Arrow Decimal128(precision, scale), we use 10^precision - 1\n        let precision = array.precision();\n        let max_value = 10_i128.pow(precision as u32) - 1;\n\n        let from_arrow = |value| {\n            const NAN: i128 = i128::MIN + 1;\n            let res = match value {\n                // Check for special values using Arrow Decimal's max value, not i128::MAX\n                NAN => Decimal::NaN,\n                v if v == max_value => Decimal::PositiveInf,\n                v if v == -max_value => Decimal::NegativeInf,\n                i128::MAX => Decimal::PositiveInf, // Fallback for old data\n                i128::MIN => Decimal::NegativeInf, // Fallback for old data\n                _ => Decimal::truncated_i128_and_scale(value, array.scale() as u32)\n                    .ok_or_else(|| ArrayError::from_arrow(\"decimal overflow\"))?,","sourceCodeStart":1565,"sourceCodeEnd":1601,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/common/src/array/arrow/arrow_impl.rs#L1565-L1601","documentation":"Converting an Arrow Decimal128 array into RisingWave's DecimalArray is only implemented for non-negative scale. Arrow (and Iceberg) can represent decimals with negative scale (value scaled by 10^|scale|), but RW decimal conversion does not support that representation, so the conversion bails.","triggerScenarios":"TryFrom<&arrow_array::Decimal128Array> for DecimalArray is invoked (primarily by the Iceberg source reading decimal columns) when array.scale() < 0, e.g. Arrow type Decimal128(5, -2).","commonSituations":"Iceberg table column defined with a negative decimal scale; a writer exported data with scaled-truncated decimals; upstream schema evolution introduced negative-scale decimals.","solutions":["Avoid negative-scale decimals in the source table schema (use scale >= 0)","Rescale the column upstream (e.g. cast to scale 0 or higher before writing/reading)","Pre-process the Arrow array with a cast to a non-negative-scale decimal type before conversion"],"exampleFix":"// before\nlet decimal = DecimalArray::try_from(arrow_decimal_array)?;\n// after\nlet casted = arrow_decimal_array.reinterpret_cast(arrow::datatypes::DataType::Decimal128(20, 0));\nlet decimal = DecimalArray::try_from(&casted)?;","handlingStrategy":"validation","validationCode":"fn check_arrow_decimal_ok(dt: &arrow::datatypes::DataType) -> Result<(), String> {\n    if let arrow::datatypes::DataType::Decimal128(p, s) = dt {\n        if *s < 0 { return Err(format!(\"negative scale {} unsupported\", s)); }\n    }\n    Ok(())\n}","typeGuard":"fn is_supported_decimal(a: &arrow_array::Decimal128Array) -> bool { a.scale() >= 0 }","tryCatchPattern":"match DecimalArray::try_from(&arrow_arr) {\n    Err(e) if e.to_string().contains(\"negative scale\") => {\n        let casted = arrow_arr.reinterpret_cast(arrow::datatypes::DataType::Decimal128(20, 0));\n        DecimalArray::try_from(&casted)\n    }\n    other => other,\n}","preventionTips":["Define Iceberg decimal columns with non-negative scale","Inspect source schemas with arrow's schema inspection before planning conversions","Cast negative-scale columns upstream during ingestion"],"tags":["arrow","decimal","iceberg","unsupported"],"backgroundTag":"unsupported-operation","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}