{"record":{"id":"f754ff3beebb40dc","repo":"rust-lang/rust-analyzer","slug":"unclosed-token-literal","errorCode":null,"errorMessage":"unclosed token literal","messagePattern":"unclosed token literal","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"lib/ungrammar/src/lexer.rs","lineNumber":88,"sourceCode":"    }\n}\n\nfn advance(input: &mut &str) -> Result<TokenKind> {\n    let mut chars = input.chars();\n    let c = chars.next().unwrap();\n    let res = match c {\n        '=' => TokenKind::Eq,\n        '*' => TokenKind::Star,\n        '?' => TokenKind::QMark,\n        '(' => TokenKind::LParen,\n        ')' => TokenKind::RParen,\n        '|' => TokenKind::Pipe,\n        ':' => TokenKind::Colon,\n        '\\'' => {\n            let mut buf = String::new();\n            loop {\n                match chars.next() {\n                    None => bail!(\"unclosed token literal\"),\n                    Some('\\\\') => match chars.next() {\n                        Some(c) if is_escapable(c) => buf.push(c),\n                        _ => bail!(\"invalid escape in token literal\"),\n                    },\n                    Some('\\'') => break,\n                    Some(c) => buf.push(c),\n                }\n            }\n            TokenKind::Token(buf)\n        }\n        c if is_ident_char(c) => {\n            let mut buf = String::new();\n            buf.push(c);\n            loop {\n                match chars.clone().next() {\n                    Some(c) if is_ident_char(c) => {\n                        chars.next();\n                        buf.push(c);","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/rust-lang/rust-analyzer/blob/e8f7e90aa3e7b26aa9a000200f606c1078da99ec/lib/ungrammar/src/lexer.rs#L70-L106","documentation":"The ungrammar lexer raises this while scanning a token literal started with a single quote (`'...'`). If the input ends before a closing quote is found, `chars.next()` returns `None` and the lexer bails. It guards against truncated or malformed grammar files.","triggerScenarios":"A `.ungrammar` file contains `'` starting a token literal that is never closed before end of file, e.g. `Foo = 'bar`.","commonSituations":"Hand-editing grammar files and deleting the closing quote; copy-paste truncation; merge conflicts leaving partial literals.","solutions":["Add the missing closing single quote to the token literal","Check the line/offset reported by the lexer to find the unterminated literal","If the quote was intentional as text, note that ungrammar only uses `'` for token literals — remove or escape it"],"exampleFix":"// before (grammar.ron / .ungrammar)\nExpr = 'binary\n// after\nExpr = 'binary'","handlingStrategy":"validation","validationCode":"// pre-check a .ungrammar file for unterminated token literals\nfn has_unterminated_literal(src: &str) -> bool {\n    src.chars().filter(|&c| c == '\\'').count() % 2 != 0\n}","typeGuard":null,"tryCatchPattern":"match ungrammar::Grammar::parse(src) {\n    Err(e) if e.to_string().contains(\"unclosed token literal\") => {\n        eprintln!(\"grammar file has an unterminated '...' literal: {e}\")\n    }\n    r => r?,\n}","preventionTips":["Count single quotes when hand-editing token literals","Run the grammar parse in CI to catch truncation early","Avoid manual edits near token literals; re-read the file after merges"],"tags":["lexer","ungrammar","syntax","parse-error"],"backgroundTag":"unclosed-string-literal","analyzedSha":"e8f7e90aa3e7b26aa9a000200f606c1078da99ec","analyzedAt":"2026-09-03T21:08:06.959Z","contentChangedAt":"2026-09-03T21:08:06.959Z","schemaVersion":2},"datasetVersion":"2026-09-11T07:07:21.782Z"}