{"record":{"id":"96a166b39300b3ad","repo":"janhq/jan","slug":"utf8error","errorCode":null,"errorMessage":"<Utf8Error>","messagePattern":"<Utf8Error>","errorType":"exception","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"src-tauri/plugins/tauri-plugin-llamacpp/src/gguf/helpers.rs","lineNumber":73,"sourceCode":"\n    let value_type_u32 = reader.read_u32::<LittleEndian>()?;\n    let value_type = GgufValueType::try_from(value_type_u32)?;\n    let value = read_gguf_value(reader, value_type)?;\n\n    Ok((key, value))\n}\n\nfn read_gguf_string<R: Read + ReadBytesExt>(reader: &mut R) -> io::Result<String> {\n    let len = reader.read_u64::<LittleEndian>()?;\n    if len > (1024 * 1024) {\n        return Err(io::Error::new(\n            io::ErrorKind::InvalidData,\n            format!(\"String length {} is unreasonably large\", len),\n        ));\n    }\n    let mut buf = vec![0u8; len as usize];\n    reader.read_exact(&mut buf)?;\n    String::from_utf8(buf).map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e))\n}\n\nfn read_gguf_value<R: Read + Seek + ReadBytesExt>(\n    reader: &mut R,\n    value_type: GgufValueType,\n) -> io::Result<String> {\n    match value_type {\n        GgufValueType::Uint8 => Ok(reader.read_u8()?.to_string()),\n        GgufValueType::Int8 => Ok(reader.read_i8()?.to_string()),\n        GgufValueType::Uint16 => Ok(reader.read_u16::<LittleEndian>()?.to_string()),\n        GgufValueType::Int16 => Ok(reader.read_i16::<LittleEndian>()?.to_string()),\n        GgufValueType::Uint32 => Ok(reader.read_u32::<LittleEndian>()?.to_string()),\n        GgufValueType::Int32 => Ok(reader.read_i32::<LittleEndian>()?.to_string()),\n        GgufValueType::Float32 => Ok(reader.read_f32::<LittleEndian>()?.to_string()),\n        GgufValueType::Bool => Ok((reader.read_u8()? != 0).to_string()),\n        GgufValueType::String => read_gguf_string(reader),\n        GgufValueType::Uint64 => Ok(reader.read_u64::<LittleEndian>()?.to_string()),\n        GgufValueType::Int64 => Ok(reader.read_i64::<LittleEndian>()?.to_string()),","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/janhq/jan/blob/fad3f12a147d138388a66f0d92a02b2675f65294/src-tauri/plugins/tauri-plugin-llamacpp/src/gguf/helpers.rs#L55-L91","documentation":"Produced by `String::from_utf8(buf)` inside `read_gguf_string` when the bytes read for a GGUF string key or value are not valid UTF-8. The `Utf8Error` is wrapped into an `io::Error` of kind `InvalidData`. GGUF strings are spec-required to be UTF-8, so this indicates either corruption or a writer that emitted raw bytes.","triggerScenarios":"The declared-length byte slice for a metadata key or string value contains invalid UTF-8 sequences (lone continuation bytes, truncated multibyte sequences). Most often the length was correct but the bytes are not textual — e.g. a binary blob was mis-tagged as a string, or the stream is misaligned and the reader is interpreting unrelated tensor/value bytes as a string.","commonSituations":"Stream misalignment after an earlier field was read with the wrong width; a quantized file whose metadata contains a non-text blob; corruption during download or disk write; an older GGUF writer that did not enforce UTF-8.","solutions":["Confirm stream alignment by re-reading the preceding value with the correct width from the GGUF spec.","Lossily inspect the bytes (`String::from_utf8_lossy`) during debugging to see whether they look like partial text (misalignment) or random bytes (corruption).","Verify the file checksum and re-download if corrupted.","If the value is genuinely non-text, report the bug to the GGUF producer; do not silently coerce."],"exampleFix":"// before\nString::from_utf8(buf).map_err(|e| io::Error::new(InvalidData, e))\n\n// after - keep lossy variant for diagnostics but still fail loudly\nString::from_utf8(buf).map_err(|e| {\n    let lossy = String::from_utf8_lossy(&buf[..e.valid_up_to().saturating_add(1)]);\n    io::Error::new(InvalidData,\n        format!(\"invalid UTF-8 at byte {}: valid_prefix={:?}\", e.valid_up_to(), lossy))\n})","handlingStrategy":"try-catch","validationCode":"null","typeGuard":"fn is_valid_metadata_key(s: &str) -> bool {\n    s.chars().all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '_')\n}","tryCatchPattern":"match read_gguf_string(reader) {\n    Ok(s) => Ok(s),\n    Err(e) if e.to_string().contains(\"Utf8Error\") || e.kind() == InvalidData => {\n        Err(CorruptFile(\"non-UTF-8 string in metadata\".into()))\n    }\n    Err(e) => Err(e.into()),\n}","preventionTips":["Treat UTF-8 failures as corruption or misalignment signals.","Lossy-decode during debugging to spot partial-text patterns.","Verify checksums before parsing."],"tags":["gguf","utf8","string","validation","llamacpp","rust"],"backgroundTag":null,"analyzedSha":"fad3f12a147d138388a66f0d92a02b2675f65294","analyzedAt":"2026-08-12T20:33:47.516Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}