{"record":{"id":"15d420c50ae67a6c","repo":"dbt-labs/dbt-core","slug":"cannot-consume-eof","errorCode":null,"errorMessage":"cannot consume EOF","messagePattern":"cannot consume EOF","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/dbt-sql/dbt-sql-utils/src/input_streams.rs","lineNumber":62,"sourceCode":"            name: \"<empty>\".to_string(),\n            data_raw,\n            index: 0,\n            // phantom: Default::default(),\n        }\n    }\n}\nimpl<Data: Deref> IntStream for CaseInsensitiveInputStream<Data>\nwhere\n    Data::Target: InputData,\n{\n    #[inline]\n    fn consume(&mut self) {\n        if let Some(index) = self.data_raw.offset(self.index, 1) {\n            self.index = index;\n            // self.current = self.data_raw.deref().item(index).unwrap_or(TOKEN_EOF);\n            // Ok(())\n        } else {\n            unreachable!(\"cannot consume EOF\");\n        }\n    }\n\n    #[inline]\n    fn la(&mut self, mut offset: isize) -> i32 {\n        assert!(offset != 0, \"offset must not be 0\");\n\n        if offset == 1 {\n            return match self.data_raw.item(self.index) {\n                Some(v) => match v {\n                    97..=122 => v - 32,\n                    _ => v,\n                },\n                None => int_stream::EOF,\n            };\n        }\n        if offset < 0 {\n            offset += 1; // e.g., translate LA(-1) to use offset i=0; then data[p+0-1]","sourceCodeStart":44,"sourceCodeEnd":80,"githubUrl":"https://github.com/dbt-labs/dbt-core/blob/0267ce9170576975b76b64ce856b2e5848e96617/crates/dbt-sql/dbt-sql-utils/src/input_streams.rs#L44-L80","documentation":"The ANTLR-style input stream's consume() advances the current index by one token via offset(). If offset() returns None the stream is already at EOF, and consuming past EOF is a lexer/parser invariant violation: no well-formed parse should ever ask to consume the EOF sentinel itself. This panic means the calling recognizer logic advanced one token too far.","triggerScenarios":"A parser/lexer loop calls consume() when la(1) is already EOF, or after recognizing a token stream that ends without the recognizer stopping at the EOF sentinel.","commonSituations":"Custom grammar rules that don't terminate on EOF; hand-written token consumption loops around dbt-sql-utils streams; changes to adaptivePredict or token production that shift the stream so EOF is reached earlier than the caller expects.","solutions":["Check la(1)/current token before calling consume(); stop consuming once the token is EOF.","Audit custom parsing loops for missing `if token == EOF { break }` guards.","Verify the input token stream is produced by the matching lexer so EOF is emitted exactly once at the end.","If reproducing on a specific SQL file, reduce the input to isolate which construct drives the parser past EOF."],"exampleFix":"// before\nwhile self.current() != TOKEN_EOF { self.consume(); }\n\n// after\nwhile let Some(tok) = self.la(1) { if tok == TOKEN_EOF { break; } self.consume(); }","handlingStrategy":"type-guard","validationCode":"if self.la(1) == Some(TOKEN_EOF) { /* do not consume */ }","typeGuard":"fn can_consume(&self) -> bool {\n    self.data_raw.offset(self.index, 1).is_some()\n}","tryCatchPattern":"// consume only when a next token exists\nif let Some(next) = self.data_raw.offset(self.index, 1) { self.index = next; } else { break; }","preventionTips":["Check la(1) != EOF before every consume() call.","In custom grammar/parse loops, break on the EOF sentinel.","Ensure the lexer emits exactly one trailing EOF token."],"tags":["parser","lexer","token-stream","unreachable","rust"],"backgroundTag":"internal-invariant-violation","analyzedSha":"0267ce9170576975b76b64ce856b2e5848e96617","analyzedAt":"2026-09-07T21:53:39.732Z","contentChangedAt":"2026-09-07T21:53:39.732Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}