risingwavelabs/risingwave · error · CommitError::Commit

commit iceberg transaction

Error message

commit iceberg transaction

What it means

Wraps an error from `Transaction::commit(catalog)` — the actual catalog commit (table metadata update) of the fast-append transaction in the Iceberg sink. Commit failures typically mean the table metadata was concurrently modified (optimistic concurrency conflict) or the catalog rejected the request.

Source

Thrown at src/connector/src/sink/iceberg/commit.rs:782

                        );
                        CommitError::Commit(anyhow!(err).context("apply iceberg fast_append"))
                    })?;

                    let table = tx.commit(catalog.as_ref()).await.map_err(|err| {
                        let err: IcebergError = err.into();
                        tracing::error!(
                            iceberg_component = "sink_committer",
                            iceberg_operation = "commit",
                            sink_id = %sink_id,
                            table = %table_name,
                            epoch,
                            snapshot_id,
                            branch = %target_branch,
                            data_file_count,
                            error = %err.as_report(),
                            "iceberg_sink_commit_transaction_failed",
                        );
                        CommitError::Commit(anyhow!(err).context("commit iceberg transaction"))
                    })?;
                    Ok(table)
                }
            },
        )
        .await
        .map_err(SinkError::Iceberg)?;
        self.table = table;

        let snapshot_num = self.table.metadata().snapshots().count();
        let catalog_name = self.config.common.catalog_name();
        let table_name = self.table.identifier().to_string();
        let metrics_labels = [&self.param.sink_name, &catalog_name, &table_name];
        GLOBAL_SINK_METRICS
            .iceberg_snapshot_num
            .with_guarded_label_values(&metrics_labels)
            .set(snapshot_num as i64);

View on GitHub (pinned to 6469eb736d)

Solutions

1. Rely on / increase commit_retry::run_with_retry retries — most conflicts are transient; check commit_retry_num config.
2. Reduce concurrent committers to the same table/branch, or ensure the catalog supports commit conflict retry (e.g., REST catalog with 409 handling).
3. Inspect the inner catalog error: fix auth (expired token), network, or catalog capacity issues as indicated.
4. If conflicts are persistent, serialize commits via a single sink or enable catalog-side retries.

Example fix

// before
let table = tx.commit(catalog.as_ref()).await.map_err(|err| {
    CommitError::Commit(anyhow!(err).context("commit iceberg transaction"))
})?;
// after: classify conflicts as retryable
let table = tx.commit(catalog.as_ref()).await.map_err(|err| {
    if is_commit_conflict(&err) { CommitError::Retryable(anyhow!(err).context("commit iceberg transaction")) }
    else { CommitError::Commit(anyhow!(err).context("commit iceberg transaction")) }
})?;

match catalog_commit {
    Err(e) if e.is_commit_conflict() => retry_with_fresh_table_load(),
    Err(e) => { log_inner_cause(&e); alert(); }
    Ok(t) => record_committed_snapshot(t),
}

When it happens

Trigger: Another writer committed a snapshot between this sink's table load and commit; catalog service returned an HTTP error (commit-conflict-retry, 409/500); authentication or catalog quota issues during the commit PUT/POST; branch requirement conflicts.

Common situations: Multiple RisingWave sink fragments or external engines (Spark, Flink) committing to the same table concurrently; REST catalog behind a load balancer with inconsistent metadata; short catalog token expiring mid-commit.

Related errors

• commit iceberg pk-index sink transaction
• `catalog.type` must be set
• Failed to list iceberg namespaces.
• Failed to load iceberg table.
• Failed to drop iceberg table.

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/53ddaff818410cac. Report an issue: GitHub.