BurntSushi/ripgrep · error · InvalidPatternError

found invalid UTF-8 in pattern at byte offset {}: {} (disabl

Error message

found invalid UTF-8 in pattern at byte offset {}: {} (disable Unicode mode and use hex escape sequences to match arbitrary bytes in a pattern, e.g., '(?-u)\xFF')

What it means

InvalidPatternError is returned by pattern_from_os / pattern_from_bytes when a regex pattern supplied by the user is not valid UTF-8. The Display message reports the byte offset where decoding failed and suggests disabling Unicode mode and using hex escapes like (?-u)\xFF so the regex engine can match the raw bytes instead. This gives end users an actionable, targeted error rather than a generic 'invalid utf8'.

Source

Thrown at crates/cli/src/pattern.rs:29

#[derive(Clone, Debug, Eq, PartialEq)]
pub struct InvalidPatternError {
    original: String,
    valid_up_to: usize,
}

impl InvalidPatternError {
    /// Returns the index in the given string up to which valid UTF-8 was
    /// verified.
    pub fn valid_up_to(&self) -> usize {
        self.valid_up_to
    }
}

impl std::error::Error for InvalidPatternError {}

impl std::fmt::Display for InvalidPatternError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        write!(
            f,
            "found invalid UTF-8 in pattern at byte offset {}: {} \
             (disable Unicode mode and use hex escape sequences to match \
             arbitrary bytes in a pattern, e.g., '(?-u)\\xFF')",
            self.valid_up_to, self.original,
        )
    }
}

impl From<InvalidPatternError> for io::Error {
    fn from(paterr: InvalidPatternError) -> io::Error {
        io::Error::new(io::ErrorKind::Other, paterr)
    }
}

/// Convert an OS string into a regular expression pattern.
///
/// This conversion fails if the given pattern is not valid UTF-8, in which

View on GitHub (pinned to 3fce3b5bb0)

Solutions

Rewrite the pattern to be valid UTF-8.
If you genuinely need to match arbitrary bytes, disable Unicode mode and use hex escapes: (?-u)\xFF in place of the offending byte.
Sanitize/normalize the pattern source (strip invalid bytes or transcode to UTF-8) before passing it to the parser.

Example fix

// before
let pat = pattern_from_bytes(b"abc\xFFxyz")?; // InvalidPatternError

// after
let pat = r"(?-u)abc\xFFxyz"; // valid UTF-8 regex matching the same bytes

Defensive patterns

Strategy: validation

Validate before calling

use std::str;
fn is_valid_utf8_pattern(b: &[u8]) -> bool {
    str::from_utf8(b).is_ok()
}

Type guard

fn valid_utf8_pattern(b: &[u8]) -> Option<&str> {
    str::from_utf8(b).ok()
}

Try / catch

let pat = match pattern_from_bytes(raw) {
    Ok(p) => p,
    Err(e) => { eprintln!("{e}"); return; }
};

Prevention

Validate patterns as UTF-8 before passing to the regex builder.
For binary patterns, build the regex with Unicode disabled and hex escapes from the start.
Normalize pattern sources (files/stdin) to UTF-8 on ingest.

When it happens

Trigger: A command-line argument pattern (OsStr) or a bytes pattern read from a file/stdin contains non-UTF-8 bytes; pattern_from_os/to_str() or pattern_from_bytes/std::str::from_utf8 returns Err.

Common situations: Searching binary logs with a pattern containing high bytes; patterns read from a file with CRLF or legacy encodings; shell that passes through raw bytes; patterns with a stray 0xFF from a copy-paste error.

Related errors

AI-assisted analysis of BurntSushi/ripgrep@3fce3b5bb0 (2026-08-06). Data as JSON: /data/errors/8ffbcb24ce27cbc4.json. Report an issue: GitHub.