{"id":"abcbe6ef1a28ef3c","repo":"BurntSushi/ripgrep","slug":"line-terminators-must-be-ascii-but-byte-is-no","errorCode":null,"errorMessage":"line terminators must be ASCII, but {byte:?} is not","messagePattern":"line terminators must be ASCII, but (.+?) is not","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"crates/regex/src/strip.rs","lineNumber":57,"sourceCode":"pub(crate) fn strip_from_match(\n    expr: Hir,\n    line_term: LineTerminator,\n) -> Result<Hir, Error> {\n    if line_term.is_crlf() {\n        let expr1 = strip_from_match_ascii(expr, b'\\r')?;\n        strip_from_match_ascii(expr1, b'\\n')\n    } else {\n        strip_from_match_ascii(expr, line_term.as_byte())\n    }\n}\n\n/// The implementation of strip_from_match. The given byte must be ASCII.\n/// This function returns an error otherwise. It also returns an error if\n/// it couldn't remove `\\n` from the given regex without leaving an empty\n/// character class in its place.\nfn strip_from_match_ascii(expr: Hir, byte: u8) -> Result<Hir, Error> {\n    if !byte.is_ascii() {\n        return Err(Error::new(ErrorKind::InvalidLineTerminator(byte)));\n    }\n    let ch = char::from(byte);\n    let invalid = || Err(Error::new(ErrorKind::NotAllowed(ch.to_string())));\n    Ok(match expr.into_kind() {\n        HirKind::Empty => Hir::empty(),\n        HirKind::Literal(hir::Literal(lit)) => {\n            if lit.iter().find(|&&b| b == byte).is_some() {\n                return invalid();\n            }\n            Hir::literal(lit)\n        }\n        HirKind::Class(hir::Class::Unicode(mut cls)) => {\n            if cls.ranges().is_empty() {\n                return Ok(Hir::class(hir::Class::Unicode(cls)));\n            }\n            let remove = hir::ClassUnicode::new(Some(\n                hir::ClassUnicodeRange::new(ch, ch),\n            ));","sourceCodeStart":39,"sourceCodeEnd":75,"githubUrl":"https://github.com/BurntSushi/ripgrep/blob/3fce3b5bb0236da2df6d99672afb8a719642eca7/crates/regex/src/strip.rs#L39-L75","documentation":"strip_from_match_ascii asserts the line terminator byte is ASCII and returns ErrorKind::InvalidLineTerminator(byte) if not. Line terminators must be a single ASCII byte (or CRLF) because the strip transform operates on byte-level character classes; a non-ASCII terminator cannot be handled by this code path.","triggerScenarios":"Constructing a regex/searcher with a LineTerminator set to a non-ASCII byte (e.g. 0xA0, a multibyte UTF-8 leader) and then running strip_from_match, which calls strip_from_match_ascii with that byte.","commonSituations":"Custom tooling that lets users pick an arbitrary line separator and a user supplies a Unicode code point > 127; misconfiguring a searcher to use a multibyte delimiter as the line terminator.","solutions":["Use an ASCII byte (or LineTerminator::crlf()) as the line terminator.","If you need a multibyte/Unicode record separator, do not route it through strip_from_match; handle splitting upstream.","Validate the chosen terminator with byte.is_ascii() before constructing the LineTerminator."],"exampleFix":"// before\nlet lt = LineTerminator::byte(0xA0); // non-ASCII -> InvalidLineTerminator\n\n// after\nlet lt = LineTerminator::byte(b'\\n'); // ASCII, supported","handlingStrategy":"validation","validationCode":"fn valid_line_terminator(byte: u8) -> bool { byte.is_ascii() }","typeGuard":"fn ascii_terminator(b: u8) -> Option<LineTerminator> {\n    if b.is_ascii() { Some(LineTerminator::byte(b)) } else { None }\n}","tryCatchPattern":"if !byte.is_ascii() {\n    eprintln!(\"line terminators must be ASCII\");\n    return;\n}","preventionTips":["Always use an ASCII byte (or CRLF) for LineTerminator.","Validate user-supplied separators with is_ascii() before constructing the terminator.","Handle multibyte record separators outside the strip-based searcher path."],"tags":["regex","line-terminator","ascii","searcher"],"analyzedSha":"3fce3b5bb0236da2df6d99672afb8a719642eca7","analyzedAt":"2026-08-06T01:39:05.073Z","schemaVersion":2}