risingwavelabs/risingwave · error · SinkError::Iceberg

schema_id should be a u64

Error message

schema_id should be a u64

What it means

Iceberg sink metadata JSON must contain a `schema_id` key holding an unsigned 64-bit integer. If the `schema_id` key exists but its JSON number cannot be represented as u64 (negative or fractional), `try_from_serialized_bytes` bails with this error.

Source

Thrown at src/connector/src/sink/iceberg/commit.rs:84

        Self::try_from_serialized_bytes(&value.metadata)
    }

    pub fn try_from_serialized_bytes(value: &[u8]) -> Result<Self> {
        let mut values = if let serde_json::Value::Object(value) =
            serde_json::from_slice::<serde_json::Value>(value)
                .context("Can't parse iceberg sink metadata")?
        {
            value
        } else {
            bail!("iceberg sink metadata should be an object");
        };

        let schema_id;
        if let Some(serde_json::Value::Number(value)) = values.remove(SCHEMA_ID) {
            schema_id = value
                .as_u64()
                .ok_or_else(|| anyhow!("schema_id should be a u64"))?;
        } else {
            bail!("iceberg sink metadata should have schema_id");
        }

        let partition_spec_id;
        if let Some(serde_json::Value::Number(value)) = values.remove(PARTITION_SPEC_ID) {
            partition_spec_id = value
                .as_u64()
                .ok_or_else(|| anyhow!("partition_spec_id should be a u64"))?;
        } else {
            bail!("iceberg sink metadata should have partition_spec_id");
        }

        let data_files: Vec<SerializedDataFile>;
        if let serde_json::Value::Array(values) = values
            .remove(DATA_FILES)
            .ok_or_else(|| anyhow!("iceberg sink metadata should have data_files object"))?
        {

View on GitHub (pinned to 6469eb736d)

Solutions

1. Fix the metadata JSON so schema_id is a non-negative integer matching the table's schema id
2. Check the writer version that produced the metadata for known serialization bugs
3. Re-commit from regenerated metadata or recreate the sink if metadata is unrecoverable
4. Validate schema_id values upstream before serializing into sink metadata

Example fix

// before (metadata JSON)
{"schema_id": -1, ...}
// after
{"schema_id": 3, ...}

if let Some(n) = json_obj.get("schema_id") {
    if n.as_u64().is_none() {
        return Err("schema_id must be a non-negative integer".into());
    }
}

fn valid_schema_id(v: &serde_json::Value) -> Option<u64> {
    v.get("schema_id").and_then(|n| n.as_u64())
}

match result {
    Err(e) if e.to_string().contains("schema_id should be a u64") => {
        // regenerate metadata or correct schema_id in stored JSON
    }
    other => other?,
}

When it happens

Trigger: Metadata JSON contains `schema_id` as a negative number or a float (e.g. -1, 1.5), so `serde_json::Number::as_u64()` returns None during Iceberg sink commit.

Common situations: Hand-crafted or corrupted metadata JSON; serialization produced by a buggy/older writer; schema_id overflow when converted from another numeric type upstream.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• iceberg sink metadata should be an object
• Can't create iceberg sink write result from empty data!
• partition_spec_id should be a u64
• error converting StreamChunk to Arrow RecordBatch: {err}
• partition spec {} not found

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/42f7ab7736fba7fb. Report an issue: GitHub.