risingwavelabs/risingwave · error · ConnectorError

invalid backfill_num_rows_per_split, must be greater than 0

Error message

invalid backfill_num_rows_per_split, must be greater than 0

What it means

The PostgreSQL CDC splitter validates `backfill_num_rows_per_split` before generating snapshot splits. A value of 0 would cause infinite/zero-size splits, so the connector rejects it up front with this error. It is an input validation error on the split-generation options.

Source

Thrown at src/connector/src/source/cdc/external/postgres.rs:190

        Ok(CdcOffset::Postgres(pg_offset))
    }

    fn snapshot_read(
        &self,
        table_name: SchemaTableName,
        start_pk: Option<OwnedRow>,
        primary_keys: Vec<String>,
        limit: u32,
    ) -> BoxStream<'_, ConnectorResult<OwnedRow>> {
        assert_eq!(table_name, self.schema_table_name);
        self.snapshot_read_inner(table_name, start_pk, primary_keys, limit)
    }

    #[try_stream(boxed, ok = CdcTableSnapshotSplit, error = ConnectorError)]
    async fn get_parallel_cdc_splits(&self, options: CdcTableSnapshotSplitOption) {
        let backfill_num_rows_per_split = options.backfill_num_rows_per_split;
        if backfill_num_rows_per_split == 0 {
            return Err(anyhow::anyhow!(
                "invalid backfill_num_rows_per_split, must be greater than 0"
            )
            .into());
        }
        if options.backfill_split_pk_column_index as usize >= self.pk_indices.len() {
            return Err(anyhow::anyhow!(format!(
                "invalid backfill_split_pk_column_index {}, out of bound",
                options.backfill_split_pk_column_index
            ))
            .into());
        }
        let split_column = self.split_column(&options);
        let row_stream = if options.backfill_as_even_splits
            && is_supported_even_split_data_type(&split_column.data_type)
        {
            // For certain types, use evenly-sized partition to optimize performance.
            tracing::info!(?self.schema_table_name, ?self.rw_schema, ?self.pk_indices, ?split_column, "Get parallel cdc table snapshot even splits.");
            self.as_even_splits(options)

View on GitHub (pinned to 6469eb736d)

Solutions

1. Set `backfill_num_rows_per_split` to a positive integer appropriate for table size (e.g. 8096 or 100000).
2. Remove the option entirely so the connector default is used, if a default exists.
3. Check the source's `WITH` clause / properties file for a stray 0 value.
4. Review release notes if the option's semantics/default changed after an upgrade.

Example fix

// before
CREATE TABLE t (...) WITH (connector = 'postgres-cdc', backfill_num_rows_per_split = '0');
// after
CREATE TABLE t (...) WITH (connector = 'postgres-cdc', backfill_num_rows_per_split = '8096');

// Validate the option before building the source
let rows_per_split: i32 = props.get("backfill_num_rows_per_split").parse()?;
if rows_per_split <= 0 { return Err("backfill_num_rows_per_split must be > 0"); }

match get_parallel_cdc_splits(opts).await {
    Err(e) if e.to_string().contains("backfill_num_rows_per_split") => {
        // correct the option and resubmit
    },
    r => r?,
}

When it happens

Trigger: Calling `get_parallel_cdc_splits` with `CdcTableSnapshotSplitOption.backfill_num_rows_per_split == 0` — typically from source property `backfill_num_rows_per_split` set to 0 or defaulting incorrectly.

Common situations: Typos in CDC source `WITH` options; setting the option to 0 expecting 'unlimited'; migrating configs between versions where the default changed; programmatic construction of split options passing 0.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• source validation failed
• invalid backfill_split_pk_column_index {}, out of bound
• The database name '{}' in FROM clause does not match the dat
• Unknown source connector: {connector_name}
• Field '{field}' is not allowed to be altered on the fly for

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/10ede3ac038d761d. Report an issue: GitHub.