What causes "failed to unescape byte literal" in rust-lang/rust?

File an ICE bug against rust-lang/rust with the exact literal and rustc commit.

failed to unescape byte literal

Error message

failed to unescape byte literal

What it means

Panic in LitKind::from_token_lit when unescape_byte returns an Err for a token::Byte literal. The surrounding comment states the lexer (cook_lexer_literal) is supposed to have already validated all escapes, so reaching this panic indicates the literal token arrived in an invalid state — a broken invariant between lexer and this conversion step. By the time from_token_lit runs, the byte literal is assumed to be well-formed.

Source

Thrown at compiler/rustc_ast/src/util/literal.rs:79

        let token::Lit { kind, symbol, suffix } = lit;
        if let Some(suffix) = suffix
            && !kind.may_have_suffix()
        {
            return Err(LitError::InvalidSuffix(suffix));
        }

        // For byte/char/string literals, chars and escapes have already been
        // checked in the lexer (in `cook_lexer_literal`). So we can assume all
        // chars and escapes are valid here.
        Ok(match kind {
            token::Bool => {
                assert!(symbol.is_bool_lit());
                LitKind::Bool(symbol == kw::True)
            }
            token::Byte => {
                return unescape_byte(symbol.as_str())
                    .map(LitKind::Byte)
                    .map_err(|_| panic!("failed to unescape byte literal"));
            }
            token::Char => {
                return unescape_char(symbol.as_str())
                    .map(LitKind::Char)
                    .map_err(|_| panic!("failed to unescape char literal"));
            }

            // There are some valid suffixes for integer and float literals,
            // so all the handling is done internally.
            token::Integer => return integer_lit(symbol, suffix),
            token::Float => return float_lit(symbol, suffix),

            token::Str => {
                // If there are no characters requiring special treatment we can
                // reuse the symbol from the token. Otherwise, we must generate a
                // new symbol because the string in the LitKind is different to the
                // string in the token.
                let s = symbol.as_str();

View on GitHub (pinned to 22057b88b0)

Solutions

File an ICE bug against rust-lang/rust with the exact literal and rustc commit.
If synthesizing literals in a proc-macro or tool, validate/unescape via the public unescape_byte API and avoid emitting malformed token::Byte tokens.
Bisect nightlies to locate the regression in the lexer-to-literal pipeline.
Reduce to the smallest b'...' expression that triggers the panic and attach to the report.

Defensive patterns

Strategy: validation

Validate before calling

// unescape_byte panics inside from_token_lit when a b'...' literal has an
// invalid escape. Validate the raw symbol BEFORE handing it to the lexer/lowering.
use std::str::FromStr;
fn valid_byte_literal(sym: &str) -> bool {
    // Strip the surrounding b'...'
    let inner = sym.strip_prefix("b'").and_then(|s| s.strip_suffix('"')).or_else(|| sym.strip_prefix("b'").and_then(|s| s.strip_suffix('\'')));
    let Some(inner) = inner else { return false; };
    rustc_lexer::unescape::unescape_byte(inner).is_ok()
}
if !valid_byte_literal(symbol.as_str()) {
    return Err(format!("invalid byte literal: {}", symbol));
}

Type guard

fn is_valid_byte_lit(kind: token::LitKind, sym: Symbol) -> bool {
    matches!(kind, token::Byte) && rustc_lexer::unescape::unescape_byte(sym.as_str()).is_ok()
}

Try / catch

let lit = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
    LitKind::from_token_lit(raw_lit)
}));
match lit {
    Ok(Ok(kind)) => kind,
    Ok(Err(e)) => return Err(format!("literal error: {:?}", e)),
    Err(_) => return Err(format!("byte literal failed to unescape: {}", raw_lit.symbol)),
}

Prevention

Restrict byte-literal escapes to the legal set: \\x00..\\x7F, \\\\, \\', \\n, \\r, \\t, \\0. Bytes >= 0x80 are not allowed in b'...'.
Never emit raw high-bit bytes or \u{...} inside a byte literal — use a u8 from_int expression instead.
In codegen that synthesizes literals, prefer constructing LitKind::Byte(value) directly rather than round-tripping through a source symbol that has to be re-unescaped.
Lint generated source for b'...' literals with a regex before feeding it to the compiler.

When it happens

Trigger: Produced when code constructs a token::Byte literal whose symbol contains an invalid escape sequence (e.g. b'\q') and bypasses lexer validation before calling from_token_lit. Also reachable via fuzzing the lexer/parser boundary or by an internal refactor that lets invalid escapes slip through cook_lexer_literal.

Common situations: Nightly rustc regressions; proc-macros that synthesize literal tokens via unstable spans/tokens APIs; fuzz harnesses targeting rustc_lexer. End-user source code with a bad byte literal yields a normal diagnostic long before this panic.

Related errors

AI-assisted analysis of rust-lang/rust@22057b88b0 (2026-08-03). Data as JSON: /data/errors/a8868bdee8e79262.json. Report an issue: GitHub.