{"record":{"id":"9e3f37166f02e431","repo":"dotnet/wpf","slug":"sr-format-sr-tokenizerhelperemptytoken-charindex-str","errorCode":null,"errorMessage":"SR.Format(SR.TokenizerHelperEmptyToken, _charIndex, _str)","messagePattern":"SR\\.Format\\(SR\\.TokenizerHelperEmptyToken, _charIndex, _str\\)","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.DotNet.Wpf/src/Shared/MS/Internal/TokenizerHelper.cs","lineNumber":240,"sourceCode":"                ++newTokenLength;\n            }\n\n            // if quoteCount isn't zero we hit the end of the string\n            // before the ending quote\n            if (quoteCount > 0)\n            {\n                throw new System.InvalidOperationException(SR.Format(SR.TokenizerHelperMissingEndQuote, _str));                \n            }\n\n            ScanToNextToken(separator); // move so at the start of the nextToken for next call\n\n            // finally made it, update the _currentToken values\n            _currentTokenIndex = newTokenIndex;\n            _currentTokenLength = newTokenLength;\n\n            if (_currentTokenLength < 1)\n            {\n                throw new System.InvalidOperationException(SR.Format(SR.TokenizerHelperEmptyToken, _charIndex, _str));\n            }\n\n            return true;\n        }\n\n        // helper to move the _charIndex to the next token or to the end of the string\n        private void ScanToNextToken(char separator)\n        {\n            // if already at end of the string don't bother\n            if (_charIndex < _strLen)\n            {\n                char currentChar = _str[_charIndex];\n\n                // check that the currentChar is a space or the separator.  If not\n                // we have an error. this can happen in the quote case\n                // that the char after the quotes string isn't a char.\n                if (!(currentChar == separator) &&\n                    !Char.IsWhiteSpace(currentChar))","sourceCodeStart":222,"sourceCodeEnd":258,"githubUrl":"https://github.com/dotnet/wpf/blob/81131a70a4c573cd62748a5c36908fc4d662daa9/src/Microsoft.DotNet.Wpf/src/Shared/MS/Internal/TokenizerHelper.cs#L222-L258","documentation":"TokenizerHelper.NextToken throws InvalidOperationException with TokenizerHelperEmptyToken when the token it extracted has length < 1, i.e. two separators appeared with nothing between them. The message includes the character index and full string.","triggerScenarios":"Parsing a token list containing consecutive separators like '1,,2' or a separator immediately followed by end-of-string, so GetCurrentToken yields an empty token.","commonSituations":"Hand-edited XAML PointCollection/DoubleCollection strings with doubled commas; programmatically joined values where an element was null/empty.","solutions":["Remove empty elements (double separators) from the input string","Filter out null/empty values before joining values into the string","Validate the token list format before calling the converter","Catch InvalidOperationException and report the index from the message"],"exampleFix":"// before\npc.Parse(\"10,20,,30,40\");\n// after\npc.Parse(\"10,20,30,40\");","handlingStrategy":"validation","validationCode":"static bool NoDoubleSeparators(string s, char sep) =>\n    !s.Contains(new string(sep, 2));","typeGuard":null,"tryCatchPattern":"try { helper.NextToken(false); }\ncatch (InvalidOperationException ex) { /* empty token at index from message */ }","preventionTips":["Filter null/empty items before joining value lists","Collapse duplicate separators with a regex pre-pass","Validate with a quick split-and-check before parsing"],"tags":["wpf","tokenizer","empty-token","parse-error"],"backgroundTag":"empty-required-field","analyzedSha":"81131a70a4c573cd62748a5c36908fc4d662daa9","analyzedAt":"2026-09-14T10:12:48.479Z","contentChangedAt":"2026-09-14T10:12:48.479Z","schemaVersion":2},"datasetVersion":"2026-09-21T21:30:21.729Z"}