stalwartlabs/stalwart · error · io::Error(InvalidData)

Invalid UTF-8

Error message

Invalid UTF-8

What it means

The JMAP log reader (`next` on a line iterator) reads raw bytes per line and converts them with String::from_utf8; any non-UTF-8 bytes produce this std::io::Error with kind InvalidData and message 'Invalid UTF-8'. It surfaces as the yielded item of the log-line iterator, i.e. the log file contains bytes that are not valid UTF-8.

Source

Thrown at crates/jmap/src/registry/mapping/log.rs:507

    /// Create a new `RawRevLines` struct from a Reader`.
    /// Internal buffering for iteration will use `cap` bytes at a time.
    pub fn with_capacity(cap: usize, reader: R) -> RevLines<R> {
        RevLines(RawRevLines::with_capacity(cap, reader))
    }
}

impl<R: Read + Seek> Iterator for RevLines<R> {
    type Item = Result<String, std::io::Error>;

    fn next(&mut self) -> Option<Result<String, std::io::Error>> {
        let line = match self.0.next_line().transpose()? {
            Ok(line) => line,
            Err(error) => return Some(Err(error)),
        };

        Some(
            String::from_utf8(line)
                .map_err(|_| std::io::Error::new(std::io::ErrorKind::InvalidData, "Invalid UTF-8")),
        )
    }
}

View on GitHub (pinned to e962003857)

Solutions

1. Locate the offending log file and inspect it with `file` / `iconv -f utf-8 -t utf-8` to find invalid byte sequences.
2. Re-encode the log to UTF-8 (e.g. `iconv -f latin1 -t utf-8`) if a non-UTF-8 encoding was used.
3. Trim/repair truncated lines (usually at file tail) or regenerate the log.
4. If you control the writer, ensure it writes UTF-8 and flushes whole lines atomically (or use from_utf8_lossy semantics when reading).

Example fix

// before
String::from_utf8(line).map_err(|_| std::io::Error::new(std::io::ErrorKind::InvalidData, "Invalid UTF-8"))
// after (caller-side lossy handling)
let text = String::from_utf8_lossy(&line);
Ok(text.into_owned())

// check a log file for invalid UTF-8 before reading
std::fs::read(path).map(|bytes| String::from_utf8(bytes).is_ok())

for line in log_lines {
    match line {
        Ok(text) => process(&text),
        Err(e) if e.kind() == std::io::ErrorKind::InvalidData => {
            tracing::warn!("skipping non-UTF-8 log line: {e}");
        }
        Err(e) => return Err(e.into()),
    }
}

When it happens

Trigger: Iterating log lines via the mapping/log.rs reader when a line contains invalid UTF-8 — e.g. binary data, partially-written multibyte characters split by truncation/rotation, or a log written in a non-UTF-8 encoding (Latin-1, GBK).

Common situations: Log files corrupted by truncation mid-multibyte-character during rotation; logs produced by processes emitting binary output; environment locale producing non-UTF-8 output redirected into the log; partially flushed writes on crash.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

AI-assisted analysis of stalwartlabs/stalwart@e962003857 (2026-09-06). Data as JSON: /api/errors/3f7ccd9c52b4d60d. Report an issue: GitHub.