risingwavelabs/risingwave · error

create iceberg catalog for pk-index sink

Error message

create iceberg catalog for pk-index sink

What it means

When the pk-index sink coordinator initializes, it builds an Iceberg catalog from the sink's `IcebergConfig` via `create_catalog()`. Any failure constructing the catalog (bad catalog type, unreachable REST/Hive/Glue endpoint, bad credentials) is wrapped with this context so the failure is attributable to pk-index sink init.

Source

Thrown at src/meta/src/manager/iceberg_pk_index_sink/coordinator.rs:374

            let f = entry.data_file();
            if let Some(delete_file) = pending.remove(f.file_path()) {
                delete_file.set_partition(f.partition().clone());
                if pending.is_empty() {
                    return Ok(());
                }
            }
        }
    }
    Ok(())
}

async fn load_catalog_and_table(
    iceberg_config: &IcebergConfig,
) -> Result<(Arc<dyn Catalog>, Table)> {
    let catalog = iceberg_config
        .create_catalog()
        .await
        .map_err(|e| anyhow!(e).context("create iceberg catalog for pk-index sink"))?;
    let table = iceberg_config
        .load_table()
        .await
        .map_err(|e| anyhow!(e).context("load iceberg table for pk-index sink"))?;
    Ok((catalog, table))
}

/// Read every persisted row for this sink, recovering `prev_committed_epoch` and pending commits.
async fn recovery(
    db: &DatabaseConnection,
    sink_id: SinkId,
) -> Result<(Option<u64>, Vec<EpochCommit>)> {
    fail::fail_point!("iceberg_v3_recovery_fail", |_| Err(anyhow::anyhow!(
        "injected: iceberg_v3_recovery_fail"
    )));
    let rows = list_sink_states_ordered_by_epoch(db, sink_id)
        .await
        .context("list pending sink states for pk-index sink recovery")?;

View on GitHub (pinned to 6469eb736d)

Solutions

  1. Verify the Iceberg connection/sink config: catalog type, URI, warehouse path, and credentials; test reachability from the meta node (curl the catalog URI).
  2. Check meta node logs for the inner error beneath this context to see the underlying catalog error (auth vs network vs unknown type).
  3. Ensure the catalog backend is running and credentials are valid/rotated; fix then restart the sink.

Example fix

// before
WITH ('catalog.uri'='http://iceberg-rest:8181/x')
// after: correct catalog uri & type
WITH ('catalog.type'='rest', 'catalog.uri'='http://iceberg-rest:8181/catalog')
Defensive patterns

Strategy: try-catch

Validate before calling

// preflight: verify catalog reachability
// curl -sSf $CATALOG_URI/v1/config > /dev/null || echo "catalog unreachable"

Try / catch

match load_catalog_and_table(&cfg).await {
    Ok((catalog, table)) => { /* proceed */ }
    Err(e) if e.to_string().contains("create iceberg catalog") => {
        tracing::error!(error = ?e, "catalog init failed; check catalog.type/uri/credentials");
    }
    Err(e) => return Err(e),
}

Prevention

When it happens

Trigger: `init` -> `load_catalog_and_table` -> `iceberg_config.create_catalog()` fails: unknown catalog type in config, network/DNS failure to the catalog service, invalid warehouse URI, missing auth credentials.

Common situations: Misconfigured `catalog.type` or `catalog.uri` on the Iceberg sink/connection; catalog service (e.g. REST server or Hive metastore) down or unreachable from the meta node; expired cloud credentials (AWS/Glue).

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/6ec203730a21b3e9. Report an issue: GitHub.