risingwavelabs/risingwave · error · SinkError

config deserialization error: {e}

Error message

config deserialization error: {e}

What it means

The GCS file sink derives GcsConfig from the WITH options BTreeMap by serializing it to JSON and deserializing into the typed struct. A missing required field, wrong value type, or unrecognized key makes serde_json::from_value fail, surfaced as SinkError::Config with "config deserialization error: {e}". The wrapped serde message pinpoints the field that could not be converted.

Source

Thrown at src/connector/src/sink/file_sink/gcs.rs:98

        let operator: Operator = Operator::new(builder)?
            .layer(LoggingLayer::default())
            .layer(RetryLayer::default());
        Ok(operator)
    }
}

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct GcsSink;

impl OpendalSinkBackend for GcsSink {
    type Properties = GcsConfig;

    const SINK_NAME: &'static str = GCS_SINK;

    fn from_btreemap(btree_map: BTreeMap<String, String>) -> Result<Self::Properties> {
        let config = serde_json::from_value::<GcsConfig>(serde_json::to_value(btree_map).unwrap())
            .map_err(|e| SinkError::Config(anyhow!(e)))?;
        if config.r#type != SINK_TYPE_APPEND_ONLY && config.r#type != SINK_TYPE_UPSERT {
            return Err(SinkError::Config(anyhow!(
                "`{}` must be {}, or {}",
                SINK_TYPE_OPTION,
                SINK_TYPE_APPEND_ONLY,
                SINK_TYPE_UPSERT
            )));
        }
        Ok(config)
    }

    fn new_operator(properties: GcsConfig) -> Result<Operator> {
        FileSink::<GcsSink>::new_gcs_sink(properties)
    }

    fn get_path(properties: Self::Properties) -> String {
        properties.common.path
    }

View on GitHub (pinned to 6469eb736d)

Solutions

1. Inspect the inner serde error for the exact offending field and fix that WITH option.
2. Provide all required gcs options: gcs_bucket_name, gcs_service_account_key, r#type, path, and format options.
3. Make sure the service account key is a complete, valid JSON credential string.
4. Remove misspelled or connector-incompatible option names (s3_*/azblob_* fields are not valid for gcs).
5. Cross-check the gcs sink docs for the current option schema in your RisingWave version.

Example fix

// before
CREATE SINK s FROM mv WITH (
  connector = 'gcs',
  bucket_name = 'mybucket',
  type = 'append-only'
);
// after
CREATE SINK s FROM mv WITH (
  connector = 'gcs',
  gcs_bucket_name = 'mybucket',
  gcs_service_account_key = '<full service account json>',
  r#type = 'append-only',
  path = 'exports/'
);

const REQUIRED: [&str; 3] = ["gcs_bucket_name", "gcs_service_account_key", "path"];
for k in REQUIRED {
    if !opts.contains_key(k) {
        return Err(format!("missing gcs sink option: {k}"));
    }
}
let key = &opts["gcs_service_account_key"];
if !key.trim_start().starts_with('{') {
    return Err("gcs_service_account_key must be a JSON service account object".to_string());
}
match opts.get("r#type").map(String::as_str) {
    Some("append-only") | Some("upsert") => {}
    other => return Err(format!("r#type must be append-only or upsert, got {other:?}")),
}

match GcsSink::from_btreemap(opts) {
    Ok(cfg) => proceed(cfg),
    Err(SinkError::Config(e)) => eprintln!("fix gcs sink WITH options: {e:#}"),
    Err(e) => return Err(e),
}

When it happens

Trigger: CREATE SINK with connector='gcs' missing required options (gcs_bucket_name, gcs_service_account_key, path, r#type, file_format), supplying a mistyped value, or passing an option key that does not exist on GcsConfig.

Common situations: Forgetting gcs_service_account_key or providing an invalid service-account JSON as the value; typo like 'bucket' instead of 'gcs_bucket_name'; copying s3/azblob options into a gcs sink; upgrades introducing new required fields.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• config deserialization error: {e}
• config deserialization error: {e}
• `{}` must be {}, or {}
• sink type unsupported: {}
• sink format unsupported: {}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/20d47201ab041545. Report an issue: GitHub.