BurntSushi/ripgrep · error · Error

line terminators must be ASCII, but {byte:?} is not

Error message

line terminators must be ASCII, but {byte:?} is not

What it means

strip_from_match_ascii asserts the line terminator byte is ASCII and returns ErrorKind::InvalidLineTerminator(byte) if not. Line terminators must be a single ASCII byte (or CRLF) because the strip transform operates on byte-level character classes; a non-ASCII terminator cannot be handled by this code path.

Source

Thrown at crates/regex/src/strip.rs:57

pub(crate) fn strip_from_match(
    expr: Hir,
    line_term: LineTerminator,
) -> Result<Hir, Error> {
    if line_term.is_crlf() {
        let expr1 = strip_from_match_ascii(expr, b'\r')?;
        strip_from_match_ascii(expr1, b'\n')
    } else {
        strip_from_match_ascii(expr, line_term.as_byte())
    }
}

/// The implementation of strip_from_match. The given byte must be ASCII.
/// This function returns an error otherwise. It also returns an error if
/// it couldn't remove `\n` from the given regex without leaving an empty
/// character class in its place.
fn strip_from_match_ascii(expr: Hir, byte: u8) -> Result<Hir, Error> {
    if !byte.is_ascii() {
        return Err(Error::new(ErrorKind::InvalidLineTerminator(byte)));
    }
    let ch = char::from(byte);
    let invalid = || Err(Error::new(ErrorKind::NotAllowed(ch.to_string())));
    Ok(match expr.into_kind() {
        HirKind::Empty => Hir::empty(),
        HirKind::Literal(hir::Literal(lit)) => {
            if lit.iter().find(|&&b| b == byte).is_some() {
                return invalid();
            }
            Hir::literal(lit)
        }
        HirKind::Class(hir::Class::Unicode(mut cls)) => {
            if cls.ranges().is_empty() {
                return Ok(Hir::class(hir::Class::Unicode(cls)));
            }
            let remove = hir::ClassUnicode::new(Some(
                hir::ClassUnicodeRange::new(ch, ch),
            ));

View on GitHub (pinned to 3fce3b5bb0)

Solutions

1. Use an ASCII byte (or LineTerminator::crlf()) as the line terminator.
2. If you need a multibyte/Unicode record separator, do not route it through strip_from_match; handle splitting upstream.
3. Validate the chosen terminator with byte.is_ascii() before constructing the LineTerminator.

Example fix

// before
let lt = LineTerminator::byte(0xA0); // non-ASCII -> InvalidLineTerminator

// after
let lt = LineTerminator::byte(b'\n'); // ASCII, supported

fn valid_line_terminator(byte: u8) -> bool { byte.is_ascii() }

fn ascii_terminator(b: u8) -> Option<LineTerminator> {
    if b.is_ascii() { Some(LineTerminator::byte(b)) } else { None }
}

if !byte.is_ascii() {
    eprintln!("line terminators must be ASCII");
    return;
}

When it happens

Trigger: Constructing a regex/searcher with a LineTerminator set to a non-ASCII byte (e.g. 0xA0, a multibyte UTF-8 leader) and then running strip_from_match, which calls strip_from_match_ascii with that byte.

Common situations: Custom tooling that lets users pick an arbitrary line separator and a user supplies a Unicode code point > 127; misconfiguring a searcher to use a multibyte delimiter as the line terminator.

Related errors

• the literal {:?} is not allowed in a regex
• found invalid UTF-8 in pattern at byte offset {}: {} (disabl
• pattern contains {byte:?} but it is impossible to match

AI-assisted analysis of BurntSushi/ripgrep@3fce3b5bb0 (2026-08-06). Data as JSON: /data/errors/abcbe6ef1a28ef3c.json. Report an issue: GitHub.