{"record":{"id":"6471c05df5facfa1","repo":"LykosAI/StabilityMatrix","slug":"invalid-token","errorCode":null,"errorMessage":"Invalid Token","messagePattern":"Invalid Token","errorType":"exception","errorClass":"PromptSyntaxError","httpStatus":null,"severity":"error","filePath":"StabilityMatrix.Avalonia/Models/Inference/Prompt.cs","lineNumber":133,"sourceCode":"        using var tokens = TokenizeResult.Tokens.Cast<IToken>().GetEnumerator();\n\n        // Maintain both token and text stacks for validation\n        var outputTokens = new Stack<IToken>();\n        var outputText = new Stack<string>();\n        var wildcardStack = new Stack<StringBuilder>();\n\n        // Store extra networks\n        var promptExtraNetworks = new List<PromptExtraNetwork>();\n\n        while (tokens.MoveNext())\n        {\n            var currentToken = tokens.Current;\n\n            // For any invalid syntax, throw\n            if (currentToken.Scopes.Any(s => s.Contains(\"invalid.illegal\")))\n            {\n                // Generic\n                throw new PromptSyntaxError(\n                    \"Invalid Token\",\n                    currentToken.StartIndex,\n                    GetSafeEndIndex(currentToken.EndIndex)\n                );\n            }\n\n            // Comments - ignore\n            if (currentToken.Scopes.Any(s => s.Contains(\"comment.line\")))\n            {\n                continue;\n            }\n\n            // Handle wildcard start\n            if (\n                processWildcards\n                && currentToken.Scopes.Contains(\"punctuation.definition.wildcard.begin.prompt\")\n            )\n            {","sourceCodeStart":115,"sourceCodeEnd":151,"githubUrl":"https://github.com/LykosAI/StabilityMatrix/blob/af93d6ef57c01cd890d7e0ad0a9ea8c9fcda3002/StabilityMatrix.Avalonia/Models/Inference/Prompt.cs#L115-L151","documentation":"The prompt tokenizer (Sublime-syntax based) flags malformed extra-network syntax with scopes like invalid.illegal; GetExtraNetworks inspects each token and throws PromptSyntaxError with the token's start/end offsets when found. It is a parse-time validation error pointing at the offending text region in the prompt.","triggerScenarios":"A prompt token contains invalid extra-network syntax — e.g. unbalanced brackets like <lora:model or stray < > — causing the tokenizer to mark scopes containing invalid.illegal.","commonSituations":"Hand-edited prompts with truncated tags; copy-pasted prompts with corrupted angle brackets; programmatic prompt assembly missing a closing bracket; wildcard/escaped characters confusing the tokenizer.","solutions":["Use the StartIndex/EndIndex from PromptSyntaxError to locate and fix the malformed token in the prompt string.","Ensure every extra-network tag has balanced < > and correct <type:name:weight> form.","Run ValidateExtraNetworks() on user input before submitting inference.","Escape or remove characters that the syntax definition marks illegal."],"exampleFix":"// before\nprompt = \"a photo <lora:detail_tuned:0.8\";   // missing '>'\n// after\nprompt = \"a photo <lora:detail_tuned:0.8>\";","handlingStrategy":"try-catch","validationCode":"// pre-check prompt text for balanced extra-network tags\nforeach (var tag in Regex.Matches(text, \"<[^>]*$\"))\n    throw new FormatException(\"Unclosed extra-network tag\");","typeGuard":"bool PromptWellFormed(string text) =>\n    text.Count(c => c == '<') == text.Count(c => c == '>');","tryCatchPattern":"try { prompt.Process(tokenizersPath); }\ncatch (PromptSyntaxError ex)\n{\n    editor.Select(ex.StartIndex, ex.EndIndex - ex.StartIndex);\n    ShowSyntaxHint(\"Invalid token at selection\");\n}","preventionTips":["Insert extra-network tags via UI helpers instead of free-typing angle brackets.","Validate prompts on input/blur in the editor before inference.","Highlight invalid tokens live using the same syntax definition the parser uses."],"tags":["prompt-syntax","parsing","validation"],"backgroundTag":"schema-validation-failed","analyzedSha":"af93d6ef57c01cd890d7e0ad0a9ea8c9fcda3002","analyzedAt":"2026-09-12T19:02:43.389Z","contentChangedAt":"2026-09-12T19:02:43.389Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}