{"record":{"id":"4da146ca141a80b7","repo":"databendlabs/databend","slug":"input-value-is-not-valid-utf-8-err","errorCode":null,"errorMessage":"input value is not valid utf-8: {err:?}","messagePattern":"input value is not valid utf-8: (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/query/config/src/toml.rs","lineNumber":44,"sourceCode":"\ntype TomlUnknownFieldHandler = Box<dyn Fn(&str) + Send + Sync + 'static>;\n\nimpl TomlIgnored {\n    pub fn new(handler: TomlUnknownFieldHandler) -> Self {\n        Self { handler }\n    }\n}\n\nimpl std::fmt::Debug for TomlIgnored {\n    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {\n        f.debug_struct(\"TomlIgnored\").finish()\n    }\n}\n\nimpl Parser for TomlIgnored {\n    fn parse<T: DeserializeOwned>(&mut self, bs: &[u8]) -> Result<T> {\n        let s = std::str::from_utf8(bs)\n            .map_err(|err| anyhow!(\"input value is not valid utf-8: {err:?}\"))?;\n        let de = toml::Deserializer::new(s);\n        let handler = &self.handler;\n        Ok(serde_ignored::deserialize(de, move |path| {\n            handler(path.to_string().as_str());\n        })?)\n    }\n}\n","sourceCodeStart":26,"sourceCodeEnd":52,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/query/config/src/toml.rs#L26-L52","documentation":"TomlIgnored::parse expects the input bytes to be valid UTF-8 before deserializing TOML with serde_ignored (to report ignored keys via the handler). If the byte slice is not valid UTF-8 it fails with \"input value is not valid utf-8: {err:?}\". TOML is a text format, so binary input cannot be parsed.","triggerScenarios":"Calling parse::<T>(bs) on TomlIgnored where bs contains non-UTF-8 bytes — e.g. a config file with binary content, a wrong-encoding file (UTF-16, Latin-1), or bytes read from the wrong source.","commonSituations":"Config files saved by editors with BOM/UTF-16 encoding; config accidentally fetched from a binary endpoint; concatenated/corrupted config bytes.","solutions":["Re-save the config file as UTF-8 (strip BOM; in vim: :set fileencoding=utf-8 and :set bomb<)","Check the file encoding with `file config.toml` and convert via iconv if needed","Verify the bytes passed to parse come from the intended text config, not a binary file","If the config came from a remote source, re-download and confirm content-type/text integrity"],"exampleFix":"// before: raw bytes straight into TOML parser\nlet cfg: Config = TomlIgnored::default().parse(&raw_bytes)?;\n// after: validate/convert encoding first\nlet text = String::from_utf8(raw_bytes)\n    .map_err(|e| anyhow!(\"config must be UTF-8: {e}\"))?;\nlet cfg: Config = TomlIgnored::default().parse(text.as_bytes())?;","handlingStrategy":"validation","validationCode":"std::str::from_utf8(bs).map_err(|e| anyhow!(\"config bytes must be UTF-8: {e}\"))?;","typeGuard":"fn is_utf8(bs: &[u8]) -> bool { std::str::from_utf8(bs).is_ok() }","tryCatchPattern":"match TomlIgnored::default().parse::<Config>(bs) {\n    Err(e) if e.to_string().contains(\"not valid utf-8\") => bail!(\"re-save config as UTF-8\"),\n    other => other,\n}","preventionTips":["Save config files as UTF-8 without BOM","Run `file config.toml` to check encoding after editing","Never concatenate binary content into config payloads"],"tags":["toml","config","encoding","utf-8"],"backgroundTag":"invalid-argument-format","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}