risingwavelabs/risingwave · error

parse iceberg config

Error message

parse iceberg config

What it means

After filling secrets, build_iceberg_config calls IcebergConfig::from_btreemap(with_secrets). If the resulting property map is missing required keys (catalog type, uri, warehouse) or has invalid values, the conversion fails and is wrapped with this context, aborting coordinator construction for the Iceberg pk-index sink.

Solutions

1. Log the resolved property keys and compare against IcebergConfig::from_btreemap's required keys (warehouse, catalog type, uri).
2. Fix the sink's WITH options via ALTER SINK or drop/recreate with the complete Iceberg property set.
3. Verify the catalog type is one supported by IcebergConfig (e.g. 'glue', 'rest', 's3', 'hadoop') and that endpoint/region/path-style values are valid.
4. Confirm secret values are non-empty after resolution; empty filled secrets can render the config unparseable.

Example fix

-- before: incomplete WITH clause
CREATE SINK s FROM mv WITH (connector='iceberg', warehouse='s3://bucket/');
-- after: full required config
CREATE SINK s FROM mv WITH (
  connector='iceberg', type='append only',
  catalog.type='rest', catalog.uri='http://iceberg-rest:8181',
  warehouse='s3://bucket/wh', s3.access.key='...', s3.secret.key='...'
);

// validate required iceberg keys before CREATE SINK
let required = ["catalog.type", "catalog.uri", "warehouse"];
for key in required {
    if !sink.properties.contains_key(key) {
        return Err(anyhow!("missing required iceberg sink property: {}", key));
    }
}

match build_iceberg_config(&sink) {
    Ok(cfg) => cfg,
    Err(e) if e.to_string().contains("parse iceberg config") => {
        // inspect resolved properties, fix WITH options, recreate sink
        log_resolved_properties(&sink);
        return Err(e);
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: build_iceberg_config receives a PbSink whose properties (plus filled secrets) do not satisfy IcebergConfig::from_btreemap — e.g. missing warehouse/catalog.uri/catalog-type keys, malformed URL, or unsupported catalog type in the sink's WITH options.

Common situations: CREATE SINK with an incomplete Iceberg WITH clause; typo'd property key so the expected config key is absent; sink properties altered after creation; secrets resolved to empty strings and rejected.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• adlsgen2.authority_host does not parse as a URL
• adlsgen2.authority_host must not contain a path component
• adlsgen2.authority_host must not contain a query or fragment
• adlsgen2: cannot configure both shared-key auth…
• adlsgen2: service-principal auth requires all three of…

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/9c59394ad9b40169. Report an issue: GitHub.

Appendix: source

Thrown at src/meta/src/manager/iceberg_pk_index_sink/mod.rs:60

        .unwrap_or(false);
    let pk_index_enabled = properties
        .get(ENABLE_PK_INDEX)
        .map(|v| v.eq_ignore_ascii_case("true"))
        .unwrap_or(false);
    connector_match && pk_index_enabled
}

/// Build an [`IcebergConfig`] from a [`PbSink`], filling secret refs along the
/// way. Used at CREATE SINK time and during recovery to (re-)register the
/// commit coordinator.
pub fn build_iceberg_config(pb_sink: &PbSink) -> anyhow::Result<IcebergConfig> {
    let properties: BTreeMap<String, String> = pb_sink.properties.clone().into_iter().collect();
    let secret_refs: BTreeMap<_, _> = pb_sink.secret_refs.clone().into_iter().collect();
    let with_secrets = LocalSecretManager::global()
        .fill_secrets(properties, secret_refs)
        .map_err(|e| anyhow!(e).context("fill secrets for iceberg"))?;
    IcebergConfig::from_btreemap(with_secrets)
        .map_err(|e| anyhow!(e).context("parse iceberg config"))
}

View on GitHub (pinned to 6469eb736d)