iceberg pk-index sink report missing metadata in aggregate_r

Error message

iceberg pk-index sink report missing metadata in aggregate_reports

What it means

Each report in `aggregate_reports` must carry a `metadata` payload (the serialized commit result from the writer or merger). A report with `metadata: None` cannot contribute file lists, so the coordinator bails. This indicates a wire-format/serialization bug rather than a user configuration problem.

Source

Thrown at src/meta/src/manager/iceberg_pk_index_sink/coordinator.rs:573

}

fn aggregate_reports(
    reports: &[PbIcebergPkIndexSinkMetadata],
) -> Result<IcebergPkIndexSinkAggResult> {
    let mut shared_schema_id: Option<i32> = None;
    let mut shared_partition_spec_id: Option<i32> = None;

    let mut data_files: Vec<SerializedDataFile> = Vec::new();
    let mut delete_files: Vec<SerializedDataFile> = Vec::new();
    let mut overwrite_files: Vec<SerializedDataFile> = Vec::new();

    if reports.is_empty() {
        bail!("no reports to aggregate for iceberg pk-index sink coordinator");
    }

    for r in reports {
        let Some(meta) = &r.metadata else {
            bail!("iceberg pk-index sink report missing metadata in aggregate_reports");
        };

        // Validate role: explicitly-Unspecified is a wire-format bug.
        let role = PbIcebergPkIndexSinkRole::try_from(r.role)
            .ok()
            .filter(|r| !matches!(r, PbIcebergPkIndexSinkRole::Unspecified))
            .ok_or_else(|| anyhow!("iceberg pk-index sink report has invalid role: {}", r.role))?;

        match role {
            PbIcebergPkIndexSinkRole::Writer => {
                let commit_result = IcebergCommitResult::try_from(meta)?;
                align_report_id(
                    commit_result.schema_id,
                    commit_result.partition_spec_id,
                    &mut shared_schema_id,
                    &mut shared_partition_spec_id,
                )?;
                data_files.extend(commit_result.data_files);

View on GitHub (pinned to 6469eb736d)

Solutions

Check node versions: ensure all stream/frontend nodes and the meta node run the same RisingWave build (protobuf compatibility).
Log/inspect the offending report's role to identify which actor type (Writer vs PositionDeleteMerger) sent metadata-less results.
Retry the epoch after the affected actor is restarted; if reproducible, fix the serialization path so metadata is always populated.

Example fix

// before: sending a stub
pb.metadata = None;
// after: always attach serialized result
pb.role = PbIcebergPkIndexSinkRole::Writer as i32;
pb.metadata = Some(iceberg_commit_result.into());

Defensive patterns

Strategy: type-guard

Validate before calling

for r in reports {
    anyhow::ensure!(r.metadata.is_some(), "report for role {} missing metadata", r.role);
}

Type guard

fn has_metadata(r: &PbIcebergPkIndexSinkMetadata) -> bool { r.metadata.is_some() }

Try / catch

let Some(meta) = &r.metadata else {
    tracing::error!(role = r.role, "pk-index sink report missing metadata");
    bail!("iceberg pk-index sink report missing metadata in aggregate_reports");
};

Prevention

Keep all RisingWave components on the same version to avoid protobuf skew.
Always populate the metadata oneof when emitting sink reports from actors.
Add a serialization round-trip test for PbIcebergPkIndexSinkMetadata.

When it happens

Trigger: A `PbIcebergPkIndexSinkMetadata` in the epoch's report list has `metadata == None` — e.g. an actor sent a role stub without its serialized `IcebergCommitResult`/`IcebergPositionDeleteCommitResult`, or proto deserialization left the oneof unset.

Common situations: Version skew between frontend/stream nodes and the meta node producing incompatible protobuf payloads; a writer bug serializing an empty message; corrupted report aggregation across the barrier.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/2760c9ad22883975. Report an issue: GitHub.