rust-lang/rust-analyzer · error

Next token must be ident

Error message

Next token must be ident

What it means

`to_parser_input` lowers a token tree into the parser's input format. Inside a macro definition, a `$var` metavariable followed by a fragment-specifier must be `Ident` (e.g. `$x:expr`); when the token after `$`-substitution is not an Ident where the lifetime/fragment path expects one, the invariant is violated and it panics. It indicates malformed macro-definition token trees reaching the converter.

Source

Thrown at crates/syntax-bridge/src/to_parser_input.rs:33

    let mut syntax_context_to_edition_cache = FxHashMap::default();
    let mut ctx_edition =
        |ctx| *syntax_context_to_edition_cache.entry(ctx).or_insert_with(|| span_to_edition(ctx));

    while !current.eof() {
        let tt = current.token_tree();

        // Check if it is lifetime
        if let Some(tt::TokenTree::Leaf(tt::Leaf::Punct(punct))) = tt
            && punct.char == '\''
        {
            current.bump();
            match current.token_tree() {
                Some(tt::TokenTree::Leaf(tt::Leaf::Ident(ident))) => {
                    res.push(LIFETIME_IDENT, ctx_edition(ident.span.ctx));
                    current.bump();
                    continue;
                }
                _ => panic!("Next token must be ident"),
            }
        }

        match tt {
            Some(tt::TokenTree::Leaf(leaf)) => {
                match leaf {
                    tt::Leaf::Literal(lit) => {
                        let kind = match lit.kind {
                            tt::LitKind::Byte => SyntaxKind::BYTE,
                            tt::LitKind::Char => SyntaxKind::CHAR,
                            tt::LitKind::Integer => SyntaxKind::INT_NUMBER,
                            tt::LitKind::Float => SyntaxKind::FLOAT_NUMBER,
                            tt::LitKind::Str | tt::LitKind::StrRaw(_) => SyntaxKind::STRING,
                            tt::LitKind::ByteStr | tt::LitKind::ByteStrRaw(_) => {
                                SyntaxKind::BYTE_STRING
                            }
                            tt::LitKind::CStr | tt::LitKind::CStrRaw(_) => SyntaxKind::C_STRING,
                            tt::LitKind::Err(_) => SyntaxKind::ERROR,

View on GitHub (pinned to e8f7e90aa3)

Solutions

1. Ensure only well-formed macro-definition token trees (valid `$name:frag` pairs) are passed to `to_parser_input`
2. Pre-validate macro definitions: after each `$` metavariable ident, require `:` + ident before conversion
3. Harden the converter to skip or recover from malformed `$` sequences instead of panicking

Example fix

// before
_ => panic!("Next token must be ident"),
// after
_ => {
    stdx::never!("Next token must be ident");
    res.push(LIFETIME_IDENT, ctx_edition(current_span.ctx));
    current.bump();
    continue;
}

// Validate macro-definition shape before lowering:
fn has_valid_fragment_specifiers(tt: &tt::Subtree) -> bool {
    // every `$ident` must be followed by ':' ident
    let mut iter = tt.token_trees.iter().peekable();
    while let Some(t) = iter.next() {
        if is_dollar_punct(t) {
            match (iter.next(), iter.peek()) {
                (Some(ident), Some(colon)) if is_ident(ident) && is_colon(colon) => { iter.next(); }
                _ => return false,
            }
        }
    }
    true
}

let input = std::panic::catch_unwind(AssertUnwindSafe(||
    to_parser_input(&token_tree)
)).unwrap_or_else(|_| ParserInput::empty()); // degrade instead of crashing the IDE

When it happens

Trigger: Calling `to_parser_input` on a token tree representing a macro_rules definition where after a `$` metavariable the next token is not an identifier (e.g. a literal, group, or punct instead of the fragment specifier like `expr`/`tt`), or where a `$`-lifetime path sees a non-ident token.

Common situations: See trigger scenarios.

Related errors

• unmatched closing delimiter from syntax fixup
• Token from lexer must be single char: token = {token:#?}
• {punct:#?} is not a valid punct
• We explicitly do not provide canonicalization API, as that i
• bad kind {other}

AI-assisted analysis of rust-lang/rust-analyzer@e8f7e90aa3 (2026-09-03). Data as JSON: /api/errors/e356da4f51e0ce7c. Report an issue: GitHub.