{"record":{"id":"5cedfca20943bf8a","repo":"risingwavelabs/risingwave","slug":"error-computing-partition-type-e","errorCode":null,"errorMessage":"error computing partition type: {e}","messagePattern":"error computing partition type: (.+?)","errorType":"exception","errorClass":"SinkError::Iceberg","httpStatus":null,"severity":"error","filePath":"src/connector/src/sink/iceberg/writer.rs","lineNumber":1030,"sourceCode":"/// error message. Shared spec-lookup mechanic for the pk-index merger, commit\n/// coordinator, and sink commit paths.\npub fn resolve_partition_spec(table: &Table, spec_id: i32) -> Result<PartitionSpecRef> {\n    table\n        .metadata()\n        .partition_spec_by_id(spec_id)\n        .cloned()\n        .ok_or_else(|| SinkError::Iceberg(anyhow!(\"partition spec {} not found\", spec_id)))\n}\n\n/// Resolves the partition [`StructType`] for the given `spec_id` against `schema`.\n///\n/// `schema` is passed explicitly (rather than read from the table) so callers can\n/// preserve their chosen schema, and `spec_id` is passed explicitly so callers can\n/// preserve their chosen spec (e.g. a file's own spec vs. the default spec).\npub fn resolve_partition_type(table: &Table, spec_id: i32, schema: &Schema) -> Result<StructType> {\n    resolve_partition_spec(table, spec_id)?\n        .partition_type(schema)\n        .map_err(|e| SinkError::Iceberg(anyhow!(e)))\n}\n\n/// Truncate large column statistics from `DataFile` BEFORE serialization.\n///\n/// This function directly modifies `DataFile`'s `lower_bounds` and `upper_bounds`\n/// to remove entries that exceed `MAX_COLUMN_STAT_SIZE`.\n///\n/// # Arguments\n/// * `data_file` - A `DataFile` to process\n///\n/// # Returns\n/// The modified `DataFile` with large statistics truncated\npub fn truncate_datafile(mut data_file: DataFile) -> DataFile {\n    // Process lower_bounds - remove entries with large values\n    data_file.lower_bounds_mut().retain(|field_id, datum| {\n        // Use to_bytes() to get the actual binary size without JSON serialization overhead\n        let size = match datum.to_bytes() {\n            Ok(bytes) => bytes.len(),","sourceCodeStart":1012,"sourceCodeEnd":1048,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/sink/iceberg/writer.rs#L1012-L1048","documentation":"resolve_partition_type converts a PartitionSpec into its partition StructType using the given schema; any iceberg-rust error from partition_type() is wrapped into this SinkError::Iceberg error. It typically means the spec's source field ids do not resolve against the provided schema.","triggerScenarios":"Calling resolve_partition_type(table, spec_id, schema) where the partition spec references fields (source ids) missing from `schema`, or the iceberg-rust partition_type computation fails on an incompatible field type (e.g. spec written against an older schema with since-changed types).","commonSituations":"Schema evolution (dropped/renamed partition columns) between when data files were written and committed; passing the wrong schema variant (e.g. current schema vs. file's schema) to the resolver; version upgrades in iceberg-rust changing validation strictness.","solutions":["Pass the schema that actually contains the partition source fields (e.g. the file's schema, not the newest table schema)","Confirm the partition columns still exist in the sink's schema after any schema evolution","Log/print the underlying iceberg error to identify which field id fails to resolve"],"exampleFix":"// before\nlet ptype = resolve_partition_type(&table, spec_id, &current_schema)?;\n// after\nlet file_schema = table.metadata().schema_by_id(file.schema_id).unwrap();\nlet ptype = resolve_partition_type(&table, spec_id, file_schema)?;","handlingStrategy":"try-catch","validationCode":"fn spec_fields_in_schema(spec_id: i32, schema: &Schema) -> bool {\n    // verify every source field id of the spec resolves in schema\n    spec.field_ids().iter().all(|id| schema.field_by_id(*id).is_some())\n}","typeGuard":null,"tryCatchPattern":"let ptype = resolve_partition_type(&table, spec_id, schema)\n    .with_context(|| format!(\"computing partition type for spec {spec_id}\"))?;","preventionTips":["Pass the schema version that contains the spec's source fields","Track schema/spec evolution: on change, verify sink state still resolves","Log the inner iceberg error to identify the failing field id"],"tags":["iceberg","schema","partition-type","sink"],"backgroundTag":"schema-validation-failed","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}