{"record":{"id":"c5223d577f05fd1e","repo":"risingwavelabs/risingwave","slug":"serde-json-deserialization-error-anyhow-e","errorCode":null,"errorMessage":"serde_json deserialization error (anyhow!(e))","messagePattern":"serde_json deserialization error \\(anyhow!\\(e\\)\\)","errorType":"exception","errorClass":"ConnectorError","httpStatus":null,"severity":"error","filePath":"src/connector/src/source/filesystem/opendal_source/batch_posix_fs_source.rs","lineNumber":55,"sourceCode":"pub struct BatchPosixFsSplit {\n    /// For batch posix fs, this is always the root directory. The reader will\n    /// scan all files in this directory.\n    pub file_path: String,\n    /// A unique identifier for the split, typically including a timestamp to force refresh.\n    pub split_id: SplitId,\n    /// Whether this split has finished reading all data (used for batch sources)\n    /// See [`BatchSourceSplit`] for details about recovery.\n    #[serde(skip)]\n    pub finished: bool,\n}\n\nimpl SplitMetaData for BatchPosixFsSplit {\n    fn id(&self) -> SplitId {\n        self.split_id.clone()\n    }\n\n    fn restore_from_json(value: JsonbVal) -> ConnectorResult<Self> {\n        serde_json::from_value(value.take()).map_err(|e| anyhow!(e).into())\n    }\n\n    fn encode_to_json(&self) -> JsonbVal {\n        serde_json::to_value(self.clone()).unwrap().into()\n    }\n\n    fn update_offset(&mut self, _last_seen_offset: String) -> ConnectorResult<()> {\n        // Batch source doesn't use offsets - each file is read completely once\n        Ok(())\n    }\n}\n\nimpl BatchSourceSplit for BatchPosixFsSplit {\n    fn finished(&self) -> bool {\n        self.finished\n    }\n\n    fn finish(&mut self) {","sourceCodeStart":37,"sourceCodeEnd":73,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/source/filesystem/opendal_source/batch_posix_fs_source.rs#L37-L73","documentation":"BatchPosixFsSplit::restore_from_json converts a JsonbVal back into a BatchPosixFsSplit using serde_json::from_value. Any failure to deserialize the JSON payload (wrong fields, wrong types, invalid shape) is wrapped with anyhow! and returned as a ConnectorResult error. This happens when the persisted split descriptor does not match the current Rust struct schema.","triggerScenarios":"SplitManager or source engine calls restore_from_json with a JsonbVal produced by encode_to_json of an older/different struct version, a hand-edited split JSON, or a JSON whose fields don't match BatchPosixFsSplit (missing/renamed fields, wrong types).","commonSituations":"Upgrading RisingWave across versions where BatchPosixFsSplit fields changed; corrupt or manually edited checkpoint state; splits persisted by a different filesystem connector variant being restored by this one.","solutions":["Inspect the JSON payload being restored and make sure it has exactly the fields BatchPosixFsSplit expects (split_id, file_path, offset etc.) with correct types.","Re-create the source / let it re-list files so splits are re-encoded with the current schema.","If upgrading versions, follow the migration path so persisted split state is compatible; otherwise drop and rebuild the MV/source.","Add #[serde(default)] or a custom Deserialize to BatchPosixFsSplit for backward compatibility if fields were added."],"exampleFix":"// before\nserde_json::from_value(value.take()).map_err(|e| anyhow!(e).into())\n// after\nserde_json::from_value(value.take())\n    .map_err(|e| anyhow!(\"failed to restore BatchPosixFsSplit from {}: {e}\", value.as_str().unwrap_or(\"json\")))\n    .into()","handlingStrategy":"try-catch","validationCode":"// Rust: validate split JSON before restoring\nfn split_json_is_valid(v: &serde_json::Value) -> bool {\n    v.is_object() && v.get(\"split_id\").map_or(false, |s| s.is_string())\n}","typeGuard":"fn is_valid_split_value(v: &JsonbVal) -> bool {\n    serde_json::from_value::<BatchPosixFsSplit>(v.clone().take()).is_ok()\n}","tryCatchPattern":"match BatchPosixFsSplit::restore_from_json(value) {\n    Ok(split) => split,\n    Err(e) => { log::warn!(\"split restore failed, re-listing: {e}\"); re_list_splits() }\n}","preventionTips":["Don't hand-edit persisted split JSON; always round-trip through encode_to_json.","Pin source schema across upgrades or re-create sources after connector version bumps.","Add serde(default) for newly added fields to keep old state restorable."],"tags":["serde","deserialization","filesystem-source","rust"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}