{"record":{"id":"baada5b00b7b43e3","repo":"windmill-labs/windmill","slug":"parsing-error","errorCode":null,"errorMessage":"Parsing error","messagePattern":"Parsing error","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"backend/parsers/windmill-parser-nu/src/lib.rs","lineNumber":22,"sourceCode":"use nu_parser::lex;\n\nuse serde_json::{json, Value};\nuse windmill_parser::{Arg, MainArgSignature, ObjectType, Typ};\n\npub fn parse_nu_signature(code: &str) -> anyhow::Result<MainArgSignature> {\n    let (tokens, ..) = lex(code.as_bytes(), 0, &[], &[], true);\n    let src = code.to_owned();\n    #[derive(Debug)]\n    enum LastToken {\n        None,\n        Def,\n        Main,\n        Args(String),\n    }\n    let mut last_token = LastToken::None;\n    for token in tokens {\n        let s = token.span;\n        let cont = src.get(s.start..s.end).ok_or(anyhow!(\"Parsing error\"))?;\n        last_token = match last_token {\n            LastToken::None if cont == \"def\" => LastToken::Def,\n            LastToken::Def if cont == \"main\" => LastToken::Main,\n            LastToken::Main => {\n                LastToken::Args(cont.get(1..(cont.len() - 1)).unwrap_or(\"Error\").to_owned())\n            }\n            LastToken::Args(_) => break,\n            _ => LastToken::None,\n        };\n    }\n\n    let LastToken::Args(args) = last_token else {\n        bail!(\"Cannot find main function.\");\n    };\n\n    let mut sig = MainArgSignature::default();\n    sig.auto_kind = None;\n","sourceCodeStart":4,"sourceCodeEnd":40,"githubUrl":"https://github.com/windmill-labs/windmill/blob/e474e8803ce2ff5c2df09a58dab51d45f5c922ca/backend/parsers/windmill-parser-nu/src/lib.rs#L4-L40","documentation":"Raised by parse_nu_signature in windmill-parser-nu. The Nu lexer produces tokens carrying byte spans into the source; the code slices `src.get(s.start..s.end)` and errors when the span does not index into the source string (returns None). This happens when the lexer's byte span is misaligned with the UTF-8 string slice — typically because the source contains multi-byte characters and a span boundary lands mid-codepoint, or because the lexer emitted an inconsistent/zero-width span.","triggerScenarios":"Calling parse_nu_signature on Nu source where the `def main` region is preceded/interleaved by multi-byte UTF-8 characters (e.g. accented letters, CJK, emoji) such that a token's byte span start..end is not a valid char boundary for str::get, causing the slice to return None. Also possible from lexer edge cases on unusual inputs that yield spans past end-of-input.","commonSituations":"A Nu script with comments or strings containing non-ASCII characters before the `def main` line; users pasting scripts from editors that include smart quotes or Unicode BOM/whitespace; Nu engine version changes altering token span semantics relative to what this hand-rolled tokenizer walk expects.","solutions":["Remove or replace non-ASCII characters (accents, emoji, CJK, smart quotes) before the `def main` signature — especially inside comments — or move `def main` to the top of the file.","ASCII-fold the offending text: rewrite `# café` as `# cafe`, replace curly quotes with straight ones.","Strip a leading UTF-8 BOM before parsing: pass `code.trim_start_matches('\\u{feff}')` into parse_nu_signature.","If the input is pure ASCII and it still fails, it's a span/lexer bug: check your nu-parser crate version against the one windmill-parser-nu was built for, and file an issue with the exact source."],"exampleFix":"// before (non-ASCII comment shifts/invalidates token byte spans)\n# naïve script — café setup 🎉\ndef main [x: int] { $x }\n\n// after (ASCII-only preamble, or strip BOM before calling)\n# naive script - cafe setup\ndef main [x: int] { $x }","handlingStrategy":"validation","validationCode":"// Ensure the source is BOM-free and ASCII-safe in the regions the lexer spans before parsing:\nfn prepare_for_nu_parser(code: &str) -> String {\n    code.trim_start_matches('\\u{feff}').to_string()\n}\nfn has_non_ascii(code: &str) -> bool {\n    code.chars().any(|c| !c.is_ascii())\n}","typeGuard":null,"tryCatchPattern":"match parse_nu_signature(&prepare_for_nu_parser(code)) {\n    Ok(sig) => sig,\n    Err(e) if e.to_string() == \"Parsing error\" => {\n        // likely non-ASCII content invalidating token spans: retry ASCII-only\n        parse_nu_signature(&code.chars().filter(|c| c.is_ascii()).collect::<String>())\n            .unwrap_or_default()\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Keep comments and strings before `def main` ASCII-only (no accents, emoji, smart quotes).","Strip a UTF-8 BOM from files before submitting them for signature parsing.","Put `def main` near the top of the file, before any non-ASCII content.","If non-ASCII is essential, test the script's signature parsing explicitly; pin nu-parser to the version windmill-parser-nu expects."],"tags":["rust","nu","parser","unicode","lexer"],"backgroundTag":"token-span-out-of-bounds","analyzedSha":"e474e8803ce2ff5c2df09a58dab51d45f5c922ca","analyzedAt":"2026-09-03T12:38:19.024Z","contentChangedAt":"2026-09-03T12:38:19.024Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}