{"record":{"id":"0f534bbe50aeacd1","repo":"risingwavelabs/risingwave","slug":"invalid-utf8-value-encoding-0","errorCode":null,"errorMessage":"Invalid UTF8 value encoding: {0}","messagePattern":"Invalid UTF8 value encoding: (.+?)","errorType":"error_code","errorClass":"ValueEncodingError","httpStatus":null,"severity":"error","filePath":"src/common/src/util/value_encoding/error.rs","lineNumber":21,"sourceCode":"// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n//     http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\nuse thiserror::Error;\n\n#[derive(Error, Debug)]\npub enum ValueEncodingError {\n    #[error(\"Invalid bool value encoding: {0}\")]\n    InvalidBoolEncoding(u8),\n    #[error(\"Invalid UTF8 value encoding: {0}\")]\n    InvalidUtf8(#[from] std::string::FromUtf8Error),\n    #[error(\"Invalid Date value encoding: days: {0}\")]\n    InvalidDateEncoding(i32),\n    #[error(\"invalid Timestamp value encoding: secs: {0} nsecs: {1}\")]\n    InvalidTimestampEncoding(i64, u32),\n    #[error(\"invalid Time value encoding: secs: {0} nano: {1}\")]\n    InvalidTimeEncoding(u32, u32),\n    #[error(\"Invalid null tag value encoding: {0}\")]\n    InvalidTagEncoding(u8),\n    #[error(\"Invalid jsonb encoding\")]\n    InvalidJsonbEncoding,\n    #[error(\"Invalid variant encoding\")]\n    InvalidVariantEncoding,\n    #[error(\"Invalid struct encoding: {0}\")]\n    InvalidStructEncoding(\n        #[source]\n        #[backtrace]\n        crate::array::ArrayError,","sourceCodeStart":3,"sourceCodeEnd":39,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/common/src/util/value_encoding/error.rs#L3-L39","documentation":"`ValueEncodingError::InvalidUtf8` is thrown when decoding a value-encoded string/VARCHAR: the encoded byte slice is converted to a Rust String, and invalid UTF-8 bytes cause `FromUtf8Error`, wrapped by this error via `#[from]`. It indicates the serialized text is not valid UTF-8, i.e. corrupt or foreign-encoded data.","triggerScenarios":"Decoding a value-encoded VARCHAR whose bytes contain invalid UTF-8 sequences (e.g. raw Latin-1/binary bytes), decoding at a wrong offset so a length-prefixed string slice covers unrelated bytes, or feeding non-UTF-8 producer data into a value-encoded string column.","commonSituations":"Ingesting binary payloads (MessagePack, protobuf) directly as text, GBK/Windows-1252 encoded input from upstream systems, corrupted Kafka messages, misaligned manual slicing of value-encoded buffers.","solutions":["Ensure upstream data is UTF-8: transcode it (e.g. to UTF-8) before writing into string columns","Validate that the byte range being decoded is a proper value-encoded string (correct length prefix/offset)","Inspect the offending bytes in the message to identify the corruption source and re-produce/re-ingest the data","Check writer/reader version compatibility of the value encoding format"],"exampleFix":"// before\nlet raw = vec![0xff, 0xfe]; // invalid UTF-8, e.g. Latin-1 text\nlet s = String::try_from_value_encoding(&raw)?;\n// after\nlet raw = \"café\".to_string().into_bytes(); // valid UTF-8\nlet s = String::try_from_value_encoding(&raw)?;","handlingStrategy":"validation","validationCode":"// validate UTF-8 before decoding as value-encoded string\nfn is_valid_utf8_encoding(bytes: &[u8]) -> bool {\n    std::str::from_utf8(bytes).is_ok()\n}\n","typeGuard":null,"tryCatchPattern":"match String::try_from_value_encoding(bytes) {\n    Ok(s) => s,\n    Err(ValueEncodingError::InvalidUtf8(e)) => {\n        // fall back to lossy decoding or reject the record\n        let lossy = String::from_utf8_lossy(bytes).into_owned();\n        return Ok(lossy);\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Transcode all upstream text to UTF-8 before ingest (Kafka connectors, UDFs)","Never cast binary payloads (protobuf/msgpack) into VARCHAR columns","Validate record boundaries/offsets before slicing encoded rows"],"tags":["value-encoding","utf8","deserialization","string"],"backgroundTag":"invalid-argument-value","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}