{"record":{"id":"f1e34119bac16636","repo":"nautechsystems/nautilus_trader","slug":"valid-utf-8-char-boundary-expected","errorCode":null,"errorMessage":"valid UTF-8 char boundary expected","messagePattern":"valid UTF-8 char boundary expected","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/common/src/logging/writer.rs","lineNumber":636,"sourceCode":"                i = end;\n            } else {\n                i += 1;\n            }\n            continue;\n        }\n\n        if bytes[i].is_ascii() {\n            if bytes[i] == b'\\n' || (bytes[i] >= b' ' && bytes[i] != b'\\x7f') {\n                out.push(bytes[i] as char);\n            }\n            i += 1;\n            continue;\n        }\n\n        let ch = s[i..]\n            .chars()\n            .next()\n            .expect(\"valid UTF-8 char boundary expected\");\n\n        if ch == '\\n' || (!ch.is_control() && ch != '\\u{7F}') {\n            out.push(ch);\n        }\n        i += ch.len_utf8();\n    }\n\n    out\n}\n\nfn ansi_escape_end(bytes: &[u8], start: usize) -> Option<usize> {\n    match bytes.get(start + 1).copied() {\n        Some(b'[') => csi_escape_end(bytes, start + 2),\n        Some(b']') => osc_escape_end(bytes, start + 2),\n        _ => None,\n    }\n}\n","sourceCodeStart":618,"sourceCodeEnd":654,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/common/src/logging/writer.rs#L618-L654","documentation":"`strip_ansi_and_nonprinting_to_string` walks a byte buffer with a manual index and slices `s[i..]` to read the next char. `.expect(\"valid UTF-8 char boundary expected\")` panics if `i` lands inside a multi-byte UTF-8 sequence, meaning the loop's index arithmetic (skipping ANSI escape sequences or control bytes) desynchronized from UTF-8 boundaries.","triggerScenarios":"Log/output lines containing multi-byte UTF-8 characters adjacent to ANSI escape sequences or non-printable bytes, where the byte-skipping logic lands mid-character; malformed input bytes treated as if valid UTF-8.","commonSituations":"Colored log output from external tools piped into the logger; binary or non-UTF-8 data written to the log stream; logs containing CJK/emoji text mixed with terminal control codes.","solutions":["Upgrade/patch: use char_indices() iteration instead of manual byte indexing so boundaries cannot be violated","Sanitize input to valid UTF-8 (String::from_utf8_lossy) before this function runs","Avoid piping raw ANSI-colored or binary output into the log writer; disable color in the producing tool","Reproduce with the offending log line and report the exact byte sequence to maintainers"],"exampleFix":"// before\nlet ch = s[i..].chars().next().expect(\"valid UTF-8 char boundary expected\");\n// after\nlet Some((_, ch)) = s[i..].chars().next() else { break }; // or iterate with char_indices\nif ch == '\\n' || (!ch.is_control() && ch != '\\u{7F}') { out.push(ch); }\ni += ch.len_utf8();","handlingStrategy":"validation","validationCode":"let sanitized = String::from_utf8_lossy(raw_bytes);\n// pass &str that is guaranteed valid UTF-8 to the writer","typeGuard":"fn is_safe_log_line(s: &str) -> bool {\n    s.is_char_boundary(0) && s.bytes().all(|b| b.is_ascii() || !b.is_ascii_control())\n}","tryCatchPattern":null,"preventionTips":["Disable ANSI color in upstream tools before piping into the logger","Validate/lossy-convert any non-UTF-8 data before it reaches the log writer","Keep the writer's byte-walking logic on char boundaries (char_indices)","Reproduce and report failing byte sequences upstream; pin to a fixed version"],"tags":["rust","panic","utf-8","logging","sanitization"],"backgroundTag":"internal-invariant-violation","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}