risingwavelabs/risingwave · error · SinkError

config deserialization error: {e}

Error message

config deserialization error: {e}

What it means

The filesystem (fs) file sink parses its FsConfig from the WITH options BTreeMap via serde_json (to_value then from_value). If a required option is missing, wrongly typed, or an unrecognized key is present, serde fails and the connector returns SinkError::Config with "config deserialization error: {e}". The inner serde error identifies the offending field.

Source

Thrown at src/connector/src/sink/file_sink/fs.rs:82

        let builder = Fs::default().root(&config.common.path);
        let operator: Operator = Operator::new(builder)?
            .layer(LoggingLayer::default())
            .layer(RetryLayer::default());
        Ok(operator)
    }
}

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct FsSink;

impl OpendalSinkBackend for FsSink {
    type Properties = FsConfig;

    const SINK_NAME: &'static str = FS_SINK;

    fn from_btreemap(btree_map: BTreeMap<String, String>) -> Result<Self::Properties> {
        let config = serde_json::from_value::<FsConfig>(serde_json::to_value(btree_map).unwrap())
            .map_err(|e| SinkError::Config(anyhow!(e)))?;
        if config.r#type != SINK_TYPE_APPEND_ONLY && config.r#type != SINK_TYPE_UPSERT {
            return Err(SinkError::Config(anyhow!(
                "`{}` must be {}, or {}",
                SINK_TYPE_OPTION,
                SINK_TYPE_APPEND_ONLY,
                SINK_TYPE_UPSERT
            )));
        }
        Ok(config)
    }

    fn new_operator(properties: FsConfig) -> Result<Operator> {
        FileSink::<FsSink>::new_fs_sink(properties)
    }

    fn get_path(properties: Self::Properties) -> String {
        properties.common.path
    }

View on GitHub (pinned to 6469eb736d)

Solutions

1. Read the serde message embedded in the error — it names the failing field — and fix that option.
2. Ensure mandatory fs options are present: path, r#type, and the file format/encode options for your data.
3. Remove or correct option names not defined on FsConfig.
4. Check format and encode combinations are valid (e.g. format='plain' with encode='json' or 'text').
5. Re-check the fs sink docs for the current expected option set after upgrading RisingWave.

Example fix

// before
CREATE SINK s FROM mv WITH (
  connector = 'fs',
  location = '/data/out',
  type = 'append-only'
);
// after
CREATE SINK s FROM mv WITH (
  connector = 'fs',
  path = '/data/out',
  r#type = 'append-only',
  format = 'plain',
  encode = 'json'
);

const REQUIRED: [&str; 2] = ["path", "r#type"];
for k in REQUIRED {
    if !opts.contains_key(k) {
        return Err(format!("missing fs sink option: {k}"));
    }
}
if let Some(fmt) = opts.get("format") {
    if fmt == "plain" && !opts.contains_key("encode") {
        return Err("format 'plain' requires an 'encode' option".to_string());
    }
}

match FsSink::from_btreemap(opts) {
    Ok(cfg) => proceed(cfg),
    Err(SinkError::Config(e)) => eprintln!("fix fs sink WITH options: {e:#}"),
    Err(e) => return Err(e),
}

When it happens

Trigger: CREATE SINK with connector='fs' where required fields such as r#type, path, or format/encode options are missing or malformed, an unknown/misspelled WITH key is passed, or a value cannot deserialize into the expected FsConfig field type.

Common situations: Omitting or misspelling the 'path' option; reusing s3/gcs option names with connector='fs'; mismatched format/encode combinations (e.g. format='plain' requires an encode); version drift where required fields changed.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• config deserialization error: {e}
• `{}` must be {}, or {}
• config deserialization error: {e}
• sink type unsupported: {}
• sink format unsupported: {}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/793afdbd070416b4. Report an issue: GitHub.