rust-lang/rust · error

Unexpected last token {last_token:?}

Error message

Unexpected last token {last_token:?}

What it means

`panic!("Unexpected last token {last_token:?}")` fires inside `to_attr_token_stream` when `break_last_token > 0` (the parser asked to split the trailing token into two — e.g., to unglue a multi-char operator like `>>` or `..=`) but the top of the token stack is not an `AttrTokenTree::Token`. Only a real `Token` can be split via `Token::break_two_token_op`; a `Delimited` or `AttrsTarget` cannot, so reaching this panic indicates the break count was set while the stack top is a non-token tree.

Source

Thrown at compiler/rustc_ast/src/tokenstream.rs:416

                stack_top.inner.push(AttrTokenTree::AttrsTarget(target))
            }
            FlatToken::Empty => {}
        }
    }

    if break_last_token > 0 {
        let last_token = stack_top.inner.pop().unwrap();
        if let AttrTokenTree::Token(last_token, spacing) = last_token {
            let (unglued, _) = last_token.kind.break_two_token_op(break_last_token).unwrap();

            // Tokens are always ASCII chars, so we can use byte arithmetic here.
            let mut first_span = last_token.span.shrink_to_lo();
            first_span =
                first_span.with_hi(first_span.lo() + rustc_span::BytePos(break_last_token));

            stack_top.inner.push(AttrTokenTree::Token(Token::new(unglued, first_span), spacing));
        } else {
            panic!("Unexpected last token {last_token:?}")
        }
    }
    AttrTokenStream::new(stack_top.inner)
}

/// Like `TokenTree`, but for `AttrTokenStream`.
#[derive(Clone, Debug, Encodable, Decodable)]
pub enum AttrTokenTree {
    Token(Token, Spacing),
    Delimited(DelimSpan, DelimSpacing, Delimiter, AttrTokenStream),
    /// Stores the attributes for an attribute target,
    /// along with the tokens for that attribute target.
    /// See `AttrsTarget` for more information
    AttrsTarget(AttrsTarget),
}

impl AttrTokenStream {
    pub fn new(tokens: Vec<AttrTokenTree>) -> AttrTokenStream {

View on GitHub (pinned to 22057b88b0)

Solutions

1. Reproduce with `RUST_BACKTRACE=1` and inspect the failing macro/attribute input — minimize it for a bug report.
2. Check that `break_last_token` is only set when the parser knows the final tree is a `Token`; reset it to 0 in paths where the stack top is `Delimited`/`AttrsTarget`.
3. Audit recent changes to `break_two_token_op` / attribute collection for off-by-one or wrong-variant assumptions.

Example fix

// before (collection logic sets break unconditionally):
break_last_token = n;

// after (only break when top is a Token):
if matches!(stack_top.inner.last(), Some(AttrTokenTree::Token(..))) {
    break_last_token = n;
} else {
    break_last_token = 0;
}

if let Some(last) = stream.last_token() {
    match &last.kind {
        TokenKind::Eof | TokenKind::Comma | TokenKind::Semi => { /* acceptable */ }
        _ => { /* unexpected; do not invoke gluing logic */ }
    }
}

fn is_glueable_last_token(t: &Token) -> bool {
    matches!(t.kind, TokenKind::Eof | TokenKind::Comma | TokenKind::Semi)
}

let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
    AttrTokenStream::create_glued_extensions(stream)
}));
if result.is_err() {
    // fall back to using the unextended stream
}

When it happens

Trigger: `break_last_token` is non-zero (set by attribute/token collection when the last token must be re-glued later) and `stack_top.inner.last()` is an `AttrTokenTree::Delimited` or `AttrTokenTree::AttrsTarget` rather than a `Token`. Triggered by attribute parsing edge cases where attributes attach to a node whose final tree is a delimited group.

Common situations: Bugs in attribute token collection (`collect_tokens`) that set `break_last_token` incorrectly. Macros placing attributes such that the collected token stream's last tree is a `Delimited`/`AttrsTarget`. Refactors to token gluing/breaking logic. Rare; usually an ICE on specific macro/attribute combinations.

Related errors

• attribute is missing tokens: {self:?}
• Should be `AttrTokenTree::Delimited`, not delim tokens: {:?}
• Attempted to encode LazyAttrTokenStream
• Attempted to decode LazyAttrTokenStream
• Attempted to compute stable hash for LazyAttrTokenStream

AI-assisted analysis of rust-lang/rust@22057b88b0 (2026-08-03). Data as JSON: /data/errors/defd47cb72832fa5.json. Report an issue: GitHub.