uutils/coreutils · error

UTF-8 error

Error message

UTF-8 error

What it means

Error "UTF-8 error" thrown in uutils/coreutils.

Source

Thrown at src/uucore/src/lib/lib.rs:573

/// Equivalent to `std::BufRead::lines` which outputs each line as a `Vec<u8>`,
/// which avoids panicking on non UTF-8 input.
fn read_byte_lines<R: std::io::Read>(
    mut buf_reader: BufReader<R>,
) -> impl Iterator<Item = error::UResult<Vec<u8>>> {
    iter::from_fn(move || {
        let mut buf = Vec::with_capacity(256);

        match buf_reader.read_until(b'\n', &mut buf) {
            Ok(0) => None,
            Err(e) => Some(Err(e.into())),
            Ok(_) => {
                // Trim (\r)\n
                if buf.ends_with(b"\n") {
                    buf.pop();
                    if buf.ends_with(b"\r") {
                        buf.pop();
                    }
                }

                Some(Ok(buf))
            }
        }
    })
}

/// Equivalent to `std::BufRead::lines` which outputs each line as an `OsString`.
///
/// On platforms where `OsString` cannot contain arbitrary bytes,
/// non-UTF8 inputs are reported as an error.
pub fn read_os_string_lines<R: std::io::Read>(
    buf_reader: BufReader<R>,
) -> impl Iterator<Item = error::UResult<OsString>> {
    read_byte_lines(buf_reader).map(|byte_line_res| byte_line_res.and_then(os_string_from_vec))
}

View on GitHub (pinned to 85295bbf78)

Solutions

1. Ensure the string data is valid UTF-8 before conversion; sanitize or lossy-convert invalid input.
2. Avoid constructing String from arbitrary byte buffers without validation.

When it happens

Trigger: Occurs when uucore encounters byte sequences that are not valid UTF-8 where UTF-8 text is required.

Common situations: Processing non-UTF-8 file names, arguments, or file contents in a UTF-8-only code path.

AI-assisted analysis of uutils/coreutils@85295bbf78 (2026-08-19). Data as JSON: /api/errors/1c718d9c0dafed0e. Report an issue: GitHub.