oxc-project/oxc · error · std::io::Error

stream did not contain valid UTF-8

Error message

stream did not contain valid UTF-8

What it means

Returned by the linter's fast read_to_string helper: the file's raw bytes were read successfully but simdutf8::basic::from_utf8 rejected them, and the failure is re-wrapped as io::ErrorKind::InvalidData with std's standard message. The input at fault is a source file whose encoding is not valid UTF-8 (e.g. Latin-1, UTF-16, or a stray invalid byte).

Source

Thrown at crates/oxc_linter/src/utils/mod.rs:152

pub fn starts_with_ignore_case(haystack: &str, prefix: &str) -> bool {
    let len = prefix.len();
    if haystack.len() < len {
        return false;
    }
    haystack.as_bytes()[..len].eq_ignore_ascii_case(prefix.as_bytes())
}

/// Reads the content of a path and returns it.
/// This function is faster than native `fs:read_to_string`.
///
/// # Errors
/// When the content of the path is not a valid UTF-8 bytes
pub fn read_to_string(path: &Path) -> io::Result<String> {
    // `simdutf8` is faster than `std::str::from_utf8` which `fs::read_to_string` uses internally
    let bytes = std::fs::read(path)?;
    if simdutf8::basic::from_utf8(&bytes).is_err() {
        // Same error as `fs::read_to_string` produces (`io::Error::INVALID_UTF8`)
        return Err(io::Error::new(
            io::ErrorKind::InvalidData,
            "stream did not contain valid UTF-8",
        ));
    }
    // SAFETY: `simdutf8` has ensured it's a valid UTF-8 string
    Ok(unsafe { String::from_utf8_unchecked(bytes) })
}

/// Read the contents of a UTF-8 encoded file directly into arena allocator.
/// Avoids intermediate allocations if file size is known in advance.
///
/// This function opens the file at `path`, reads its entire contents into memory
/// allocated from the given [`Allocator`], validates that the bytes are valid UTF-8,
/// and returns a borrowed `&str` pointing to the allocator-backed data.
///
/// This is useful for performance-critical workflows where zero-copy string handling is desired,
/// such as parsing large source files in memory-constrained or throughput-sensitive environments.
///

View on GitHub (pinned to e1e7af627c)

Solutions

1. Re-encode or save the file as UTF-8
2. Convert the file lossily (e.g. String::from_utf8_lossy) if replacement characters are acceptable
3. Detect a UTF-16 BOM and transcode before validation

When it happens

Trigger: Thrown at crates/oxc_linter/src/utils/mod.rs:152 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of oxc-project/oxc@e1e7af627c (2026-08-20). Data as JSON: /api/errors/eaac3022ecac0115. Report an issue: GitHub.