{"record":{"id":"4d053b8230fbad08","repo":"dotnet/wpf","slug":"sr-format-sr-tokenizerhelperextradataencountered-charindex","errorCode":null,"errorMessage":"SR.Format(SR.TokenizerHelperExtraDataEncountered, _charIndex, _str)","messagePattern":"SR\\.Format\\(SR\\.TokenizerHelperExtraDataEncountered, _charIndex, _str\\)","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.DotNet.Wpf/src/Shared/MS/Internal/TokenizerHelper.cs","lineNumber":102,"sourceCode":"        internal string GetCurrentToken()\n        {\n            // if no current token, return null\n            if (_currentTokenIndex < 0)\n            {\n                return null;\n            }\n\n            return _str.Substring(_currentTokenIndex,_currentTokenLength);\n        }\n        \n        /// <summary>\n        /// Throws an exception if there is any non-whitespace left un-parsed.\n        /// </summary>\n        internal void LastTokenRequired()\n        {\n            if (_charIndex != _strLen)\n            {\n                throw new System.InvalidOperationException(SR.Format(SR.TokenizerHelperExtraDataEncountered, _charIndex, _str));\n            }\n        }\n\n        /// <summary>\n        /// Advances to the NextToken\n        /// </summary>\n        /// <returns>true if next token was found, false if at end of string</returns>\n        internal bool NextToken()\n        {\n            return NextToken(false);\n        }\n\n        /// <summary>\n        /// Advances to the NextToken, throwing an exception if not present\n        /// </summary>\n        /// <returns>The next token found</returns>\n        internal string NextTokenRequired()\n        {","sourceCodeStart":84,"sourceCodeEnd":120,"githubUrl":"https://github.com/dotnet/wpf/blob/81131a70a4c573cd62748a5c36908fc4d662daa9/src/Microsoft.DotNet.Wpf/src/Shared/MS/Internal/TokenizerHelper.cs#L84-L120","documentation":"TokenizerHelper.LastTokenRequired is called after parsing the final expected token of a formatted string (via Parse) and asserts that the entire input was consumed; any leftover non-whitespace text throws InvalidOperationException(SR.Format(SR.TokenizerHelperExtraDataEncountered, _charIndex, _str)) with the character index and full string. WPF uses this when parsing XAML attribute values such as Point, Size, Color, or matrix strings.","triggerScenarios":"Passing a string with extra trailing content to a Parse method that uses TokenizerHelper — e.g. \"1,2 3,4 extra\", a doubled comma like \"10,,20\", or an extra coordinate in a Point/Vector/Matrix string.","commonSituations":"Hand-edited XAML attribute values; strings built by concatenation that leave a trailing separator; locale-independent parsing of user-entered geometry data with more tokens than the type accepts.","solutions":["Remove any extra tokens/characters so the string contains exactly the expected number of comma/space-separated values for the type (e.g. Point needs exactly 2 numbers).","Check for accidental doubled separators (\",,\" or \" ,\") and trailing separators before parsing.","Trim and normalize the string (collapse whitespace) before calling Parse.","Wrap Parse in try/catch for InvalidOperationException and surface a friendly message including the offending position.","Use TryParse-style alternatives or pre-validate the token count with str.Split(',') where available."],"exampleFix":"// before\nvar p = (Point)TypeConverter Point.Parse(\"10,20,30\"); // extra token\n// after\nstring s = \"10,20,30\";\nvar parts = s.Split(new[] { ',', ' ' }, StringSplitOptions.RemoveEmptyEntries);\nif (parts.Length != 2)\n    throw new FormatException($\"Point requires exactly 2 values, got {parts.Length}: '{s}'\");\nvar p = Point.Parse(string.Join(\",\", parts.Take(2)));","handlingStrategy":"validation","validationCode":"// ensure exactly the expected number of tokens before parsing\nstring[] tokens = str.Split(new[] { ',', ' ' }, StringSplitOptions.RemoveEmptyEntries);\nif (tokens.Length != expectedTokenCount)\n    throw new FormatException($\"Expected {expectedTokenCount} values, got {tokens.Length}: '{str}'\");\nvar value = Parse(str);","typeGuard":"static bool HasExactTokens(string s, int n) =>\n    s.Split(new[] { ',', ' ' }, StringSplitOptions.RemoveEmptyEntries).Length == n;","tryCatchPattern":"try\n{\n    var point = Point.Parse(\"10,20,30\");\n}\ncatch (InvalidOperationException ex)\n{\n    // extra data after the last expected token\n    Console.WriteLine($\"Bad formatted value: {ex.Message}\");\n}","preventionTips":["Verify token counts against the type's expected arity (Point=2, Size=2, Matrix=6, etc.).","Reject doubled or trailing separators before parsing.","Trim/collapse whitespace in user-supplied or concatenated format strings.","Never hand-edit XAML geometry attribute strings without re-parsing them in a test."],"tags":["parse-error","tokenizer","xaml","invalid-operation"],"backgroundTag":"invalid-argument-format","analyzedSha":"81131a70a4c573cd62748a5c36908fc4d662daa9","analyzedAt":"2026-09-14T10:12:48.479Z","contentChangedAt":"2026-09-14T10:12:48.479Z","schemaVersion":2},"datasetVersion":"2026-09-21T21:30:21.729Z"}