risingwavelabs/risingwave · error · anyhow::Error

please set the separator in the with option, when there are

Error message

please set the separator in the with option, when there are multiple primary key values

What it means

When the sink has multiple primary key columns, the formatter joins their values into a document key using a delimiter. If no `separator` (delimiter) was provided in the WITH options, construction fails with this message, because a multi-column key cannot be serialized unambiguously without one.

Source

Thrown at src/connector/src/sink/elasticsearch_opensearch/elasticsearch_opensearch_formatter.rs:79

                .name;
            format!("{{{}}}", name)
        } else if pk_indices.len() == 1 {
            let index = *pk_indices.get(0).unwrap();
            let name = &schema
                .fields()
                .get(index)
                .ok_or_else(|| {
                    SinkError::ElasticSearchOpenSearch(anyhow!(
                        "no value find in sink schema, index is {:?}",
                        index
                    ))
                })?
                .name;
            format!("{{{}}}", name)
        } else {
            let delimiter = delimiter
                .as_ref()
                .ok_or_else(|| anyhow!("please set the separator in the with option, when there are multiple primary key values"))?
                .clone();
            let mut names = Vec::with_capacity(pk_indices.len());
            for index in &pk_indices {
                names.push(format!(
                    "{{{}}}",
                    schema
                        .fields()
                        .get(*index)
                        .ok_or_else(|| {
                            SinkError::ElasticSearchOpenSearch(anyhow!(
                                "no value find in sink schema, index is {:?}",
                                index
                            ))
                        })?
                        .name
                ));
            }
            names.join(&delimiter)

View on GitHub (pinned to 6469eb736d)

Solutions

1. Add `separator = '<delim>'` to the WITH clause (e.g. separator = '_')
2. Ensure the delimiter does not appear in the pk column values, to avoid key collisions
3. Alternatively reduce to a single primary key column to avoid needing a separator
4. Recreate the sink with the corrected WITH options

Example fix

// before
CREATE SINK s FROM mv WITH (connector='elasticsearch', primary_key='user_id, item_id');
// after
CREATE SINK s FROM mv WITH (connector='elasticsearch', primary_key='user_id, item_id', separator='_');

fn check_separator_for_composite_pk(with_opts: &Options, pk_len: usize) -> Result<(), String> {
    if pk_len > 1 && with_opts.get("separator").is_none() {
        return Err("multi-column primary key requires 'separator' in WITH options".into());
    }
    Ok(())
}

When it happens

Trigger: Creating an Elasticsearch/OpenSearch sink with 2+ primary key columns and no `separator` option in the WITH clause; checked in the formatter's `new` when `delimiter` is None in the multi-pk branch.

Common situations: Sinks with composite primary keys where the author added `primary_key = 'a, b'` but forgot `separator = ','`; copying single-pk sink examples to a composite-key scenario.

Understand the failure class

Background: "Must pass :limit option" / "Missing required option" — required option errors explained — this error's family across 41 libraries.

Related errors

• Primary key not defined for upsert doris sink (please define
• no value find in sink schema, index is {:?}
• Primary key columns not found. Please set the `primary_key`
• `{}` must be {}, or {}
• Can't get fe host from url

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/1a08cdb2fc1b8749. Report an issue: GitHub.