Should be `AttrTokenTree::Delimited`, not delim tokens: {:?}

Error message

Should be `AttrTokenTree::Delimited`, not delim tokens: {:?}

What it means

`panic!("Should be \`AttrTokenTree::Delimited\`, not delim tokens: {:?}")` fires inside `MetaItem::from_tokens` when the attribute-token stream contains a bare `TokenTree::Token` whose kind is a delimiter (`(`, `[`, `{`) instead of the expected `AttrTokenTree::Delimited` wrapper. The attribute parser canonicalizes delimiters into the `Delimited` variant; encountering a raw delim token means the stream was assembled incorrectly (a non-canonical token stream).

Source

Thrown at compiler/rustc_ast/src/attr/mod.rs:526

                    };
                    iter.next();
                }
                let span = span.with_hi(segments.last().unwrap().ident.span.hi());
                Path { span, segments }
            }
            Some(TokenTree::Delimited(
                _span,
                _spacing,
                Delimiter::Invisible(InvisibleOrigin::MetaVar(
                    MetaVarKind::Meta { .. } | MetaVarKind::Path,
                )),
                _stream,
            )) => {
                // This path is currently unreachable in the test suite.
                unreachable!()
            }
            Some(TokenTree::Token(Token { kind, .. }, _)) if kind.is_delim() => {
                panic!("Should be `AttrTokenTree::Delimited`, not delim tokens: {:?}", tt);
            }
            _ => return None,
        };
        let list_closing_paren_pos = iter.peek().map(|tt| tt.span().hi());
        let kind = MetaItemKind::from_tokens(iter)?;
        let hi = match &kind {
            MetaItemKind::NameValue(lit) => lit.span.hi(),
            MetaItemKind::List(..) => list_closing_paren_pos.unwrap_or(path.span.hi()),
            _ => path.span.hi(),
        };
        let span = path.span.with_hi(hi);
        // FIXME: This parses `unsafe()` not as unsafe attribute syntax in `MetaItem`,
        // but as a parenthesized list. This (and likely `MetaItem`) should be changed in
        // such a way that builtin macros don't accept extraneous `unsafe()`.
        Some(MetaItem { unsafety: Safety::Default, path, kind, span })
    }
}

View on GitHub (pinned to 22057b88b0)

Solutions

When building an `AttrTokenStream`, always wrap matched delimiters in `AttrTokenTree::Delimited(delim_span, delim_spacing, delimiter, inner_stream)`.
Re-run the failing case under the attribute parser's token collector rather than constructing the stream manually.
Audit recent changes to token collection / `break_last_token` logic that may have left delimiters ungrouped.

Example fix

// before
stream.push(AttrTokenTree::Token(Token::open_paren(span), Spacing::Alone));

// after
stream.push(AttrTokenTree::Delimited(
    DelimSpan::single(span),
    DelimSpacing::new(Spacing::Alone, Spacing::Alone),
    Delimiter::Parenthesis,
    inner_stream,
));

Defensive patterns

Strategy: type-guard

Validate before calling

if let Some(TokenTree::Delimited(..)) = iter.peek() {
    // safe to descend into delimited group
} else if let Some(TokenTree::Token(t, _)) = iter.peek() {
    if t.kind.is_delim() {
        // malformed: delim token instead of Delimited; skip or report
    }
}

Type guard

fn is_delimited_tt(tt: &AttrTokenTree) -> bool {
    matches!(tt, AttrTokenTree::Delimited(..))
}

Prevention

The panic indicates a stream that did not round-trip through `Delimited` reconstruction — treat it as corrupt input.
Before consuming a `MetaItem`, assert the open token is a `TokenTree::Delimited`.
When constructing `AttrTokenStream`s programmatically, always wrap groups in `AttrTokenTree::Delimited`, never bare delim tokens.

When it happens

Trigger: `MetaItem::from_attr_token_stream` walks the stream and finds `Some(TokenTree::Token(Token { kind, .. }, _))` where `kind.is_delim()` is true. Triggered when code constructs an `AttrTokenStream` by pushing raw delimiter tokens instead of wrapping them as `AttrTokenTree::Delimited`.

Common situations: Internal AST/tokenstream tooling that hand-builds attribute token streams. Bugs in token collection (`collect_tokens`) that fail to group matching delimiters. Macros that emit raw `(` / `[` / `{` tokens into attribute positions. Not reproducible from ordinary source; only via non-canonical streams.

Related errors

AI-assisted analysis of rust-lang/rust@22057b88b0 (2026-08-03). Data as JSON: /data/errors/89e2a3fda6c37dcf.json. Report an issue: GitHub.