risingwavelabs/risingwave · error

Must specify 'connector' in WITH clause

Error message

Must specify 'connector' in WITH clause

What it means

ConnectorProperties::extract requires the WITH clause of a CREATE SOURCE / SINK to contain the `connector` property (UPSTREAM_SOURCE_KEY), which names the connector type. The key is absent from the provided properties map, so the type cannot be determined and extraction fails.

Source

Thrown at src/connector/src/source/base.rs:672

}

impl ConnectorProperties {
    /// Creates typed source properties from the raw `WITH` properties.
    ///
    /// It checks the `connector` field, and them dispatches to the corresponding type's `try_from_btreemap` method.
    ///
    /// `deny_unknown_fields`: Since `WITH` options are persisted in meta, we do not deny unknown fields when restoring from
    /// existing data to avoid breaking backwards compatibility. We only deny unknown fields when creating new sources.
    pub fn extract(
        with_properties: WithOptionsSecResolved,
        deny_unknown_fields: bool,
    ) -> Result<Self> {
        let (options, secret_refs) = with_properties.into_parts();
        let mut options_with_secret =
            LocalSecretManager::global().fill_secrets(options, secret_refs)?;
        let connector = options_with_secret
            .remove(UPSTREAM_SOURCE_KEY)
            .ok_or_else(|| anyhow!("Must specify 'connector' in WITH clause"))?
            .to_lowercase();
        match_source_name_str!(
            connector.as_str(),
            PropType,
            PropType::try_from_btreemap(options_with_secret, deny_unknown_fields)
                .map(ConnectorProperties::from),
            |other| bail!("connector '{}' is not supported", other)
        )
    }

    pub fn enforce_secret_source(
        with_properties: &impl WithPropertiesExt,
    ) -> crate::error::ConnectorResult<()> {
        let connector = with_properties
            .get_connector()
            .ok_or_else(|| anyhow!("Must specify 'connector' in WITH clause"))?
            .to_lowercase();
        let key_iter = with_properties.key_iter();

View on GitHub (pinned to 6469eb736d)

Solutions

1. Add connector='<name>' to the WITH clause of the CREATE SOURCE/SINK statement
2. If constructing properties in code, insert UPSTREAM_SOURCE_KEY ("connector") into the options map before calling extract
3. Check the option name spelling (it must literally be `connector`)

Example fix

// before
CREATE SOURCE s (...) WITH (topic='events');
// after
CREATE SOURCE s (...) WITH (connector='kafka', topic='events');

assert!(with_options.contains_key("connector"), "WITH clause must include connector='<name>'");

When it happens

Trigger: Calling ConnectorProperties::extract on a with-properties BTreeMap lacking the `connector` key; SQL like CREATE SOURCE ... WITH (topic='x') omitting connector='kafka'.

Common situations: Hand-building properties programmatically and forgetting the connector key; SQL that specifies only topic/broker options; renaming/refactor dropping the key.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• connector '{}' is not supported
• unsupported sink connector {}
• Unknown fields in the WITH clause: {:?}
• Must specify 'connector' in WITH clause
• connector '{}' is not supported

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/95a5f73cebefdeea. Report an issue: GitHub.