risingwavelabs/risingwave · error · SinkError::Iceberg

error from iceberg library

Error message

error from iceberg library: {err} (DefaultLocationGenerator::new failed)

What it means

DefaultLocationGenerator::new(table.metadata()) failed inside the iceberg-rs library while building the append-only writer; the library error is wrapped in SinkError::Iceberg with the note '(DefaultLocationGenerator::new failed)'. The location generator derives data file paths from table metadata and can fail when metadata is malformed for path generation (e.g. invalid table location).

Solutions

Verify the Iceberg table's `location` in metadata is an absolute, well-formed URI (s3://, gs://, file:// etc.) and fix the table or the catalog configuration
Print the underlying library error (it is inside SinkError::Iceberg) to see the exact reason DefaultLocationGenerator rejected the metadata
If on an older iceberg-rs version, check known bugs in DefaultLocationGenerator and upgrade the crate
Re-create the table with a correct location and point the RisingWave sink at the new table

Example fix

// before
DefaultLocationGenerator::new(table.metadata())
    .map_err(|err| SinkError::Iceberg(anyhow!(err)))?,
// after
DefaultLocationGenerator::new(table.metadata())
    .map_err(|err| SinkError::Iceberg(anyhow!(
        "DefaultLocationGenerator::new failed: {err}; table location = {}",
        table.metadata().location()
    )))?

Defensive patterns

Strategy: validation

Validate before calling

let loc = table.metadata().location();
if !loc.starts_with("s3://") && !loc.starts_with("gs://") && !loc.starts_with("file://") && !loc.starts_with("/") {
    return Err(format!("invalid table location: {loc}"));
}

Try / catch

let location_generator = DefaultLocationGenerator::new(table.metadata())
    .map_err(|err| SinkError::Iceberg(anyhow!("DefaultLocationGenerator::new failed: {err} (table location: {})", table.metadata().location())))?;

Prevention

Verify the Iceberg table location URI is absolute and well-formed before creating the sink
Check catalog/warehouse configuration in the environment the sink runs in
Pin iceberg-rs and review its DefaultLocationGenerator validation rules on upgrades
Print the inner library error rather than discarding it — it names the exact cause

When it happens

Trigger: Calling build_append_only when the Iceberg table's metadata carries a location that the DefaultLocationGenerator rejects — typically an empty, relative, or otherwise invalid table URI.

Common situations: Table created against a misconfigured warehouse/object-store path; table location containing characters the path generator cannot encode; a table registered with a location that is not a valid URL (e.g. missing scheme).

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/b7838377adb3f1da. Report an issue: GitHub.

Appendix: source

Thrown at src/connector/src/sink/iceberg/writer.rs:396

            target_file_size_mb = config.target_file_size_mb(),
            parquet_compression = ?config.get_parquet_compression(),
            "iceberg_sink_writer_initialized",
        );

        let parquet_writer_properties = WriterProperties::builder()
            .set_compression(config.get_parquet_compression())
            .set_max_row_group_bytes(config.write_parquet_max_row_group_bytes())
            .set_created_by(PARQUET_CREATED_BY.to_owned())
            .build();

        let parquet_writer_builder =
            ParquetWriterBuilder::new(parquet_writer_properties, schema.clone());
        let rolling_builder = RollingFileWriterBuilder::new(
            parquet_writer_builder,
            (config.target_file_size_mb() * 1024 * 1024) as usize,
            table.file_io().clone(),
            DefaultLocationGenerator::new(table.metadata())
                .map_err(|err| SinkError::Iceberg(anyhow!(err)))?,
            DefaultFileNameGenerator::new(
                writer_param.actor_id.to_string(),
                Some(unique_uuid_suffix.to_string()),
                iceberg::spec::DataFileFormat::Parquet,
            ),
        );
        let data_file_builder = DataFileWriterBuilder::new(rolling_builder);
        let monitored_builder = MonitoredGeneralWriterBuilder::new(
            data_file_builder,
            write_qps.clone(),
            write_latency.clone(),
        );
        let writer_builder = TaskWriterBuilderWrapper::new(
            monitored_builder,
            fanout_enabled,
            schema.clone(),
            partition_spec.clone(),
            true,

View on GitHub (pinned to 6469eb736d)