rust-lang/rust-analyzer · error

expected at least one token

Error message

expected at least one token

What it means

In syntax-bridge's token-collector, when a buffered token is flushed to the sink it combines the spans of all collected text ranges and calls .expect("expected at least one token") on the combined span. The invariant is that flush is only called after at least one span was pushed; hitting the panic means the token map and buffer got out of sync — text was buffered with no recorded span.

Source

Thrown at crates/syntax-bridge/src/lib.rs:1011

                    }
                    None => {
                        let parent = self.cursor.end();
                        match delim_to_str(parent.delimiter.kind, true) {
                            Some(it) => (it, parent.delimiter.close),
                            None => continue,
                        }
                    }
                };
            };
            self.buf += text;
            self.text_pos += TextSize::of(text);
            combined_span = match combined_span {
                None => Some(span),
                Some(prev_span) => Some(Self::merge_spans(prev_span, span)),
            }
        }

        self.token_map.push(self.text_pos, combined_span.expect("expected at least one token"));
        self.inner.token(kind, self.buf.as_str());
        self.buf.clear();
        // FIXME: Emitting whitespace for this is really just a hack, we should get rid of it.
        // Add whitespace between adjoint puncts
        if let Some([tt::Leaf::Punct(curr), tt::Leaf::Punct(next)]) = last_two {
            // Note: We always assume the semi-colon would be the last token in
            // other parts of RA such that we don't add whitespace here.
            //
            // When `next` is a `Punct` of `'`, that's a part of a lifetime identifier so we don't
            // need to add whitespace either.
            if curr.spacing == tt::Spacing::Alone && curr.char != ';' && next.char != '\'' {
                self.inner.token(WHITESPACE, " ");
                self.text_pos += TextSize::of(' ');
                self.token_map.push(self.text_pos, curr.span);
            }
        }
    }

View on GitHub (pinned to e8f7e90aa3)

Solutions

1. Inspect the proc-macro/token-tree input being converted; fix the producer so every emitted token has a span.
2. Update rust-analyzer/proc-macro server versions to match — older versions emitted spans the collector didn't record.
3. Reduce the failing input to a minimal token tree and file a bug with the reproduction if the input looks valid.

// Before converting, ensure every tt::TokenTree you feed carries a span
fn has_spans(tt: &tt::TopSubtree) -> bool {
    let mut ok = true;
    tt.top_subtree().token_trees().for_each(|t| match t.leaf() {
        Some(leaf) => ok &= leaf.span().range != None.into(),
        None => {}
    });
    ok
}

// Bridge panics via expect; isolate conversion
let result = std::panic::catch_unwind(AssertUnwindSafe(|| {
    syntax_bridge::syntax_tree_to_token_tree(...) 
}));
match result {
    Ok(tree) => tree,
    Err(_) => report_bridge_bug(input),
}

When it happens

Trigger: Feeding a token tree into the converter where a flush (via token()/push with kind) occurs with an empty span list — i.e., malformed or degenerate proc-macro/token-tree input where no text position range was recorded before the flush at line 1011.

Common situations: Converting output of buggy proc macros into syntax trees; token trees containing empty or zero-width tokens; version mismatches between proc-macro server and client producing unexpected tt streams.

Related errors

• bad kind {other}
• bad spacing {other}
• bad tag: {other}
• Next token must be ident
• {punct:#?} is not a valid punct

AI-assisted analysis of rust-lang/rust-analyzer@e8f7e90aa3 (2026-09-03). Data as JSON: /api/errors/a9298f7e13dd6d0f. Report an issue: GitHub.