{"record":{"id":"a88237de58e3076d","repo":"sxyazi/yazi","slug":"invalid-wtf-8-sequence","errorCode":null,"errorMessage":"Invalid WTF-8 sequence","messagePattern":"Invalid WTF-8 sequence","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"yazi-shim/src/wtf8/wtf8.rs","lineNumber":41,"sourceCode":"\n// --- FromWtf8\npub trait FromWtf8 {\n\tfn from_wtf8(wtf8: &[u8]) -> Result<&Self>;\n}\n\nimpl FromWtf8 for OsStr {\n\tfn from_wtf8(wtf8: &[u8]) -> Result<&Self> {\n\t\t#[cfg(unix)]\n\t\t{\n\t\t\tuse std::os::unix::ffi::OsStrExt;\n\t\t\tOk(Self::from_bytes(wtf8))\n\t\t}\n\t\t#[cfg(windows)]\n\t\t{\n\t\t\tif super::valid_wtf8(wtf8) {\n\t\t\t\tOk(unsafe { Self::from_encoded_bytes_unchecked(wtf8) })\n\t\t\t} else {\n\t\t\t\tErr(anyhow::anyhow!(\"Invalid WTF-8 sequence\"))\n\t\t\t}\n\t\t}\n\t}\n}\n\nimpl FromWtf8 for std::path::Path {\n\tfn from_wtf8(wtf8: &[u8]) -> Result<&Self> { Ok(OsStr::from_wtf8(wtf8)?.as_ref()) }\n}\n\n// --- FromWtf8Vec\npub trait FromWtf8Vec {\n\tfn from_wtf8_vec(wtf8: Vec<u8>) -> Result<Self>\n\twhere\n\t\tSelf: Sized;\n}\n\nimpl FromWtf8Vec for OsString {\n\tfn from_wtf8_vec(wtf8: Vec<u8>) -> Result<Self> {","sourceCodeStart":23,"sourceCodeEnd":59,"githubUrl":"https://github.com/sxyazi/yazi/blob/5f901b886b14de1f17460b6e52e9de5d67f8aba9/yazi-shim/src/wtf8/wtf8.rs#L23-L59","documentation":"`from_wtf8` on the windows target converts raw WTF-8 bytes (the UTF-8 superset that can encode unpaired surrogates used in Windows paths) into a path string, but first validates the bytes with `super::valid_wtf8`. Invalid byte sequences — e.g. malformed surrogate encodings or bad UTF-8 — are rejected with this error rather than producing a corrupted path.","triggerScenarios":"Calling `FromWtf8::from_wtf8` with a byte buffer that fails `valid_wtf8` on Windows: bytes are neither valid UTF-8 nor valid WTF-8 (e.g. an encoded unpaired surrogate written incorrectly, or truncated multi-byte sequence).","commonSituations":"Receiving file names over IPC or from FFI buffers on Windows and constructing paths from raw bytes; data corrupted in transit; hand-crafting WTF-8 for testing unpaired-surrogate paths.","solutions":["Validate the byte source — re-encode from UTF-16 with the proper surrogate algorithm before conversion","Sanitize or replace unpaired surrogates with U+FFFD before calling from_wtf8","Check for truncation: ensure the buffer contains complete multi-byte sequences","If data comes from Win32 APIs, use the correct wide-to-WTF-8 conversion helpers"],"exampleFix":"// before\nWtf8Buf::from_wtf8(raw_bytes)?; // raw_bytes has a truncated 3-byte surrogate\n// after\nlet fixed = String::from_utf8_lossy(&raw_bytes).into_owned(); // sanitize first\nlet path = Wtf8Buf::from_wtf8(fixed.into_bytes())?;","handlingStrategy":"validation","validationCode":"fn is_valid_wtf8(b: &[u8]) -> bool {\n    std::str::from_utf8(b).is_ok() // fast path; full check lives in yazi_shim::wtf8::valid_wtf8\n        || yazi_shim::wtf8::valid_wtf8(b)\n}\nif !is_valid_wtf8(&bytes) { sanitize(&mut bytes); }","typeGuard":null,"tryCatchPattern":"match Wtf8Buf::from_wtf8(bytes) {\n    Err(e) if e.to_string() == \"Invalid WTF-8 sequence\" => sanitize_and_retry(bytes),\n    other => other,\n}","preventionTips":["Convert Win32 UTF-16 names with a proven wide->WTF-8 converter","Never hand-assemble surrogate byte sequences","Sanitize untrusted bytes (U+FFFD replacement) before path construction","Check for truncation when chunking buffers"],"tags":["windows","encoding","wtf8","path"],"backgroundTag":"invalid-encoding","analyzedSha":"5f901b886b14de1f17460b6e52e9de5d67f8aba9","analyzedAt":"2026-09-02T18:38:25.566Z","contentChangedAt":"2026-09-02T18:38:25.566Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}