risingwavelabs/risingwave · error · ConnectorError

Failed to parse source definition SQL

Error message

Failed to parse source definition SQL

What it means

During `ALTER SOURCE ... ALTER_CONNECTOR_PROPS`, the meta node re-parses the source's stored definition SQL (src/meta/src/controller/streaming_job.rs:2938-2945) so it can rewrite the WITH options. If `Parser::parse_sql` fails on that stored definition, the parse error is wrapped with context 'Failed to parse source definition SQL' and returned as a Connector MetaError. This means the catalog contains a definition string that is no longer valid SQL (corruption, manual DB edit, or an upgrade that changed the SQL dialect/parser).

Source

Thrown at src/meta/src/controller/streaming_job.rs:2941

            options_with_secret
        );
        // check if the alter-ed props are valid for each Connector
        let _ = ConnectorProperties::extract(options_with_secret.clone(), true)?;
        // todo: validate via source manager

        let mut associate_table_id = None;

        // can be source_id or table_id
        // if updating an associated source, the preferred_id is the table_id
        // otherwise, it is the source_id
        let mut preferred_id = source_id.as_object_id();
        let rewrite_sql = {
            let definition = source.definition.clone();

            let [mut stmt]: [_; 1] = Parser::parse_sql(&definition)
                .map_err(|e| {
                    MetaError::from(MetaErrorInner::Connector(ConnectorError::from(
                        anyhow!(e).context("Failed to parse source definition SQL"),
                    )))
                })?
                .try_into()
                .unwrap();

            /// Formats SQL options with secret values properly resolved
            ///
            /// This function processes configuration options that may contain sensitive data:
            /// - Plaintext options are directly converted to `SqlOption`
            /// - Secret options are retrieved from the database and formatted as "SECRET {name}"
            ///   without exposing the actual secret value
            ///
            /// # Arguments
            /// * `txn` - Database transaction for retrieving secrets
            /// * `options_with_secret` - Container of options with both plaintext and secret values
            ///
            /// # Returns
            /// * `MetaResult<Vec<SqlOption>>` - List of formatted SQL options or error

View on GitHub (pinned to 6469eb736d)

Solutions

1. Run `SHOW SOURCE <name>` / `SELECT definition FROM ...` and inspect the stored definition for syntax problems.
2. Recreate the source: DROP SOURCE and CREATE SOURCE with fresh, valid SQL, then re-apply the ALTER.
3. Check whether the definition was hand-modified in the metadata DB and restore it to the statement originally issued by CREATE SOURCE.
4. If a version upgrade introduced the incompatibility, upgrade/roll back consistently or re-create affected sources.

Example fix

-- before: ALTER SOURCE src ALTER_CONNECTOR_PROPS ... fails because catalog definition is corrupt
-- after
DROP SOURCE src;
CREATE SOURCE src (...) WITH (connector = 'kafka', ...);
ALTER SOURCE src ALTER_CONNECTOR_PROPS (properties.add = 'x');

-- before altering, verify the stored definition parses
SHOW SOURCE my_source; -- confirm the definition is a valid single CREATE SOURCE statement

match client.alter_source_connector_props(...).await {
    Err(e) if e.to_string().contains("Failed to parse source definition SQL") => {
        // recover by dropping and recreating the source
    }
    other => other?,
}

When it happens

Trigger: Calling ALTER SOURCE / alter_connector_props on a source whose `definition` column fails to parse into exactly one SQL statement (zero or multiple statements, or syntax the parser rejects).

Common situations: Definitions hand-edited directly in the meta catalog database; definitions stored by an older RisingWave version whose syntax the current parser no longer accepts; truncated/corrupted catalog rows after a failed migration or restore.

Understand the failure class

Background: "query failed", "%w: SQL error" — wrapped database query errors in Go libraries explained — this error's family across 3 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• SinkError::Config(anyhow!(e))
• {object_type} not found: {name}
• expect `CREATE TABLE` or `CREATE SOURCE` statement, found: `
• Catalog error: {0}
• catalog error: {0}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/64925d8bec502778. Report an issue: GitHub.