risingwavelabs/risingwave · error · SinkError::Iceberg

partition_spec_id should be a u64

Error message

partition_spec_id should be a u64

What it means

In try_from_serialized_bytes, the 'partition_spec_id' JSON field is matched as serde_json::Value::Number, then narrowed with as_u64(). If the value is a JSON number that is not representable as u64 (negative or fractional), this error fires. It enforces that partition_spec_id is a non-negative integer before it is cast to i32.

Source

Thrown at src/connector/src/sink/iceberg/commit.rs:93

            value
        } else {
            bail!("iceberg sink metadata should be an object");
        };

        let schema_id;
        if let Some(serde_json::Value::Number(value)) = values.remove(SCHEMA_ID) {
            schema_id = value
                .as_u64()
                .ok_or_else(|| anyhow!("schema_id should be a u64"))?;
        } else {
            bail!("iceberg sink metadata should have schema_id");
        }

        let partition_spec_id;
        if let Some(serde_json::Value::Number(value)) = values.remove(PARTITION_SPEC_ID) {
            partition_spec_id = value
                .as_u64()
                .ok_or_else(|| anyhow!("partition_spec_id should be a u64"))?;
        } else {
            bail!("iceberg sink metadata should have partition_spec_id");
        }

        let data_files: Vec<SerializedDataFile>;
        if let serde_json::Value::Array(values) = values
            .remove(DATA_FILES)
            .ok_or_else(|| anyhow!("iceberg sink metadata should have data_files object"))?
        {
            data_files = values
                .into_iter()
                .map(from_value::<SerializedDataFile>)
                .collect::<std::result::Result<_, _>>()
                .unwrap();
        } else {
            bail!("iceberg sink metadata should have data_files object");
        }

View on GitHub (pinned to 6469eb736d)

Solutions

  1. Fix the metadata producer so partition_spec_id is serialized via serde_json::Value::Number from a i32/u32 value
  2. Validate the raw JSON: partition_spec_id must be an integer >= 0 and <= i32::MAX (it is cast to i32 afterwards)
  3. Replace the corrupted metadata entry with a freshly produced write result

Example fix

// before
{"schema_id":1,"partition_spec_id":-1,"data_files":[]}
// after
{"schema_id":1,"partition_spec_id":0,"data_files":[]}
Defensive patterns

Strategy: validation

Validate before calling

fn valid_partition_spec_id(bytes: &[u8]) -> bool {
    serde_json::from_slice::<serde_json::Value>(bytes).ok()
        .and_then(|v| v.get("partition_spec_id").cloned())
        .map_or(false, |v| v.as_u64().map_or(false, |n| n <= i32::MAX as u64))
}

Type guard

fn as_nonneg_int(v: &serde_json::Value, key: &str) -> Option<i64> {
    v.get(key)?.as_i64().filter(|n| *n >= 0)
}

Try / catch

match try_from_serialized_bytes(&bytes) {
    Ok(r) => r,
    Err(e) => return Err(SinkError::Iceberg(anyhow!("invalid metadata: {}", e.as_report()))),
}

Prevention

When it happens

Trigger: Metadata JSON where 'partition_spec_id' is a negative integer (e.g. -1) or a float (e.g. 0.5); happens when metadata was produced by non-standard code or manually edited.

Common situations: Manually patched metadata in a debugging session, third-party tooling writing Iceberg sink metadata, corrupt values after storage truncation.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/c6cda8881e1a1132. Report an issue: GitHub.