rust-lang/rust · error

attribute is missing tokens: {self:?}

Error message

attribute is missing tokens: {self:?}

What it means

`panic!("attribute is missing tokens: {self:?}")` is fired by `Attribute::token_trees()` when an `AttrKind::Normal` attribute has `normal.tokens == None`. The invariant is that a fully-parsed Normal attribute always carries a lazy token stream; a missing stream means the attribute was constructed incompletely (tokens were never populated during parsing or were dropped by a transformation).

Source

Thrown at compiler/rustc_ast/src/attr/mod.rs:299

            AttrKind::Normal(normal) => normal.item.meta(self.span),
            AttrKind::Synthetic(..) | AttrKind::DocComment(..) => None,
        }
    }

    pub fn meta_kind(&self) -> Option<MetaItemKind> {
        match &self.kind {
            AttrKind::Normal(normal) => normal.item.meta_kind(),
            AttrKind::Synthetic(..) => unreachable!(),
            AttrKind::DocComment(..) => None,
        }
    }

    pub fn token_trees(&self) -> Vec<TokenTree> {
        match self.kind {
            AttrKind::Normal(ref normal) => normal
                .tokens
                .as_ref()
                .unwrap_or_else(|| panic!("attribute is missing tokens: {self:?}"))
                .to_attr_token_stream()
                .to_token_trees(),
            // Empty tokens here ensures synthetic attributes are invisible to proc macros.
            AttrKind::Synthetic(..) => vec![],
            AttrKind::DocComment(comment_kind, data) => vec![TokenTree::token_alone(
                token::DocComment(comment_kind, self.style, data),
                self.span,
            )],
        }
    }

    pub fn deprecation_note(&self) -> Option<Ident> {
        match &self.kind {
            AttrKind::Normal(normal) if normal.item.path == sym::deprecated => {
                let meta = &normal.item;

                // #[deprecated = "..."]
                if let Some(s) = meta.value_str() {

View on GitHub (pinned to 22057b88b0)

Solutions

1. When constructing a `Normal` attribute, always populate `tokens` (e.g., via `AttrAnnotatedTokenStream` / `dummy_attr_tokens` helpers).
2. If you mutate attributes, propagate the original token stream rather than discarding it.
3. Reproduce with `RUST_BACKTRACE=1` to find which constructor left `tokens` as `None`, then fix that construction site.

Example fix

// before
Attribute { kind: AttrKind::Normal(NormalAttr { item, tokens: None }), .. }

// after
Attribute {
    kind: AttrKind::Normal(NormalAttr {
        item,
        tokens: Some(LazyAttrTokenStream::new(...)),
    }),
    ..
}

if let AttrKind::Normal(ref normal) = attr.kind {
    if normal.tokens.is_none() {
        // tokens not yet attached; do not call attr.token_trees()
    }
}

fn attr_has_tokens(attr: &Attribute) -> bool {
    match attr.kind {
        AttrKind::Normal(ref n) => n.tokens.is_some(),
        AttrKind::Synthetic(_) | AttrKind::DocComment(..) => true,
    }
}

When it happens

Trigger: Calling `attr.token_trees()` (or any path that reads `.tokens` on a `Normal` attribute) on an `Attribute` whose `NormalAttr.tokens` field is `None`. Happens when AST is hand-built or mutated (e.g., by a macro or a tool that constructs `Attribute` structs directly) without setting the `tokens` field.

Common situations: Procedural macros or rustc internals that build `Attribute` values via constructors that don't seed `tokens`. Refactors that move attribute creation and forget the token stream. Inconsistent deserialization paths that drop the tokens field. Rare in user code; common in tooling that synthesizes attributes.

Related errors

• empty prefix in a simple import
• Should be `AttrTokenTree::Delimited`, not delim tokens: {:?}
• if we had an explicit self, we wouldn't be here
• `UseTree::ident` can only be used on a simple import
• cloning statement `NodeId`s is prohibited by default, the vi

AI-assisted analysis of rust-lang/rust@22057b88b0 (2026-08-03). Data as JSON: /data/errors/2ab0b932e68d5bff.json. Report an issue: GitHub.