nautechsystems/nautilus_trader · error

valid UTF-8 char boundary expected

Error message

valid UTF-8 char boundary expected

What it means

`strip_ansi_and_nonprinting_to_string` walks a byte buffer with a manual index and slices `s[i..]` to read the next char. `.expect("valid UTF-8 char boundary expected")` panics if `i` lands inside a multi-byte UTF-8 sequence, meaning the loop's index arithmetic (skipping ANSI escape sequences or control bytes) desynchronized from UTF-8 boundaries.

Source

Thrown at crates/common/src/logging/writer.rs:636

                i = end;
            } else {
                i += 1;
            }
            continue;
        }

        if bytes[i].is_ascii() {
            if bytes[i] == b'\n' || (bytes[i] >= b' ' && bytes[i] != b'\x7f') {
                out.push(bytes[i] as char);
            }
            i += 1;
            continue;
        }

        let ch = s[i..]
            .chars()
            .next()
            .expect("valid UTF-8 char boundary expected");

        if ch == '\n' || (!ch.is_control() && ch != '\u{7F}') {
            out.push(ch);
        }
        i += ch.len_utf8();
    }

    out
}

fn ansi_escape_end(bytes: &[u8], start: usize) -> Option<usize> {
    match bytes.get(start + 1).copied() {
        Some(b'[') => csi_escape_end(bytes, start + 2),
        Some(b']') => osc_escape_end(bytes, start + 2),
        _ => None,
    }
}

View on GitHub (pinned to 18893faf8b)

Solutions

1. Upgrade/patch: use char_indices() iteration instead of manual byte indexing so boundaries cannot be violated
2. Sanitize input to valid UTF-8 (String::from_utf8_lossy) before this function runs
3. Avoid piping raw ANSI-colored or binary output into the log writer; disable color in the producing tool
4. Reproduce with the offending log line and report the exact byte sequence to maintainers

Example fix

// before
let ch = s[i..].chars().next().expect("valid UTF-8 char boundary expected");
// after
let Some((_, ch)) = s[i..].chars().next() else { break }; // or iterate with char_indices
if ch == '\n' || (!ch.is_control() && ch != '\u{7F}') { out.push(ch); }
i += ch.len_utf8();

let sanitized = String::from_utf8_lossy(raw_bytes);
// pass &str that is guaranteed valid UTF-8 to the writer

fn is_safe_log_line(s: &str) -> bool {
    s.is_char_boundary(0) && s.bytes().all(|b| b.is_ascii() || !b.is_ascii_control())
}

When it happens

Trigger: Log/output lines containing multi-byte UTF-8 characters adjacent to ANSI escape sequences or non-printable bytes, where the byte-skipping logic lands mid-character; malformed input bytes treated as if valid UTF-8.

Common situations: Colored log output from external tools piped into the logger; binary or non-UTF-8 data written to the log stream; logs containing CJK/emoji text mixed with terminal control codes.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• API key is valid UTF-8
• in-flight mutex poisoned
• wallet balance mutex poisoned
• instrument update lock poisoned
• rate limiter decision lock poisoned

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/f1e34119bac16636. Report an issue: GitHub.