risingwavelabs/risingwave · error · ConnectorError

invalid backfill_split_pk_column_index {}, out of bound

Error message

invalid backfill_split_pk_column_index {}, out of bound

What it means

The PostgreSQL CDC splitter also validates `backfill_split_pk_column_index` against the table's primary key indices. If the index is >= the number of PK columns, it cannot designate a split column, so this error is thrown. It is an out-of-bounds validation on the split options.

Source

Thrown at src/connector/src/source/cdc/external/postgres.rs:196

        start_pk: Option<OwnedRow>,
        primary_keys: Vec<String>,
        limit: u32,
    ) -> BoxStream<'_, ConnectorResult<OwnedRow>> {
        assert_eq!(table_name, self.schema_table_name);
        self.snapshot_read_inner(table_name, start_pk, primary_keys, limit)
    }

    #[try_stream(boxed, ok = CdcTableSnapshotSplit, error = ConnectorError)]
    async fn get_parallel_cdc_splits(&self, options: CdcTableSnapshotSplitOption) {
        let backfill_num_rows_per_split = options.backfill_num_rows_per_split;
        if backfill_num_rows_per_split == 0 {
            return Err(anyhow::anyhow!(
                "invalid backfill_num_rows_per_split, must be greater than 0"
            )
            .into());
        }
        if options.backfill_split_pk_column_index as usize >= self.pk_indices.len() {
            return Err(anyhow::anyhow!(format!(
                "invalid backfill_split_pk_column_index {}, out of bound",
                options.backfill_split_pk_column_index
            ))
            .into());
        }
        let split_column = self.split_column(&options);
        let row_stream = if options.backfill_as_even_splits
            && is_supported_even_split_data_type(&split_column.data_type)
        {
            // For certain types, use evenly-sized partition to optimize performance.
            tracing::info!(?self.schema_table_name, ?self.rw_schema, ?self.pk_indices, ?split_column, "Get parallel cdc table snapshot even splits.");
            self.as_even_splits(options)
        } else {
            tracing::info!(?self.schema_table_name, ?self.rw_schema, ?self.pk_indices, ?split_column, "Get parallel cdc table snapshot uneven splits.");
            self.as_uneven_splits(options)
        };
        pin_mut!(row_stream);
        #[for_await]

View on GitHub (pinned to 6469eb736d)

Solutions

1. Set `backfill_split_pk_column_index` to 0 (the first PK column) unless you know the PK layout.
2. Confirm the upstream table's PK column count and use an index within range (0-based).
3. Drop and re-create the CDC source so split options and PK metadata are consistent.
4. Remove the option so the connector picks the default split column.

Example fix

// before: single-column PK but index 1 configured
WITH (connector = 'postgres-cdc', backfill_split_pk_column_index = '1');
// after
WITH (connector = 'postgres-cdc', backfill_split_pk_column_index = '0');

// Check index against the actual PK column count
let pk_count = pk_indices.len() as i32;
if opts.backfill_split_pk_column_index >= pk_count || opts.backfill_split_pk_column_index < 0 {
    return Err("backfill_split_pk_column_index out of bounds");
}

match get_parallel_cdc_splits(opts).await {
    Err(e) if e.to_string().contains("out of bound") => {
        // reset the index to 0 and retry
    },
    r => r?,
}

When it happens

Trigger: Calling `get_parallel_cdc_splits` with `options.backfill_split_pk_column_index` greater than or equal to `self.pk_indices.len()` — e.g. index 1 supplied for a single-column PK.

Common situations: Hand-editing CDC properties with a PK column index that doesn't match the table's actual PK; upstream PK reduced to fewer columns after the source was configured; off-by-one assumptions (1-based vs 0-based index).

Related errors

• invalid backfill_num_rows_per_split, must be greater than 0
• PostgreSQL table {} exists, but the connection user `{}` doe
• Postgres table should define the primary key for non-append-
• {:?} data type is not supported
• {connector_name} snapshot primary-key index {index} is out o

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/369f50be6e0e31b9. Report an issue: GitHub.