{"record":{"id":"421b2197e098113a","repo":"dotnet/machinelearning","slug":"replace-normalizer-string-pattern-must-be-a-stri","errorCode":null,"errorMessage":"Replace normalizer 'String' pattern must be a string.","messagePattern":"Replace normalizer 'String' pattern must be a string\\.","errorType":"validation","errorClass":"InvalidDataException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.Tokenizers/Normalizer/SentencePieceNormalizationStep.cs","lineNumber":428,"sourceCode":"                _literal = literal;\n                _regex = regex;\n                _content = content;\n            }\n\n            public static ReplaceStep Create(JsonElement normalizer)\n            {\n                string content = normalizer.TryGetProperty(\"content\", out JsonElement contentElement) && contentElement.ValueKind == JsonValueKind.String\n                    ? contentElement.GetString() ?? \"\" : \"\";\n                if (!normalizer.TryGetProperty(\"pattern\", out JsonElement pattern) || pattern.ValueKind != JsonValueKind.Object)\n                {\n                    throw new InvalidDataException(\"Replace normalizer is missing its pattern.\");\n                }\n\n                if (pattern.TryGetProperty(\"String\", out JsonElement literal))\n                {\n                    if (literal.ValueKind != JsonValueKind.String)\n                    {\n                        throw new InvalidDataException(\"Replace normalizer 'String' pattern must be a string.\");\n                    }\n\n                    return new ReplaceStep(literal.GetString() ?? \"\", regex: null, content);\n                }\n\n                if (pattern.TryGetProperty(\"Regex\", out JsonElement regex))\n                {\n                    if (regex.ValueKind != JsonValueKind.String)\n                    {\n                        throw new InvalidDataException(\"Replace normalizer 'Regex' pattern must be a string.\");\n                    }\n\n                    string regexPattern = regex.GetString()!;\n                    try\n                    {\n                        return new ReplaceStep(literal: null, new Regex(regexPattern, RegexOptions.CultureInvariant, _regexTimeout), content);\n                    }\n                    catch (ArgumentException ex)","sourceCodeStart":410,"sourceCodeEnd":446,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.Tokenizers/Normalizer/SentencePieceNormalizationStep.cs#L410-L446","documentation":"If a Replace normalizer's pattern object has a 'String' member, its value must be a JSON string — the literal substring to replace. Any other JSON kind (number, null, object) triggers this InvalidDataException.","triggerScenarios":"tokenizer.json contains {\"type\":\"Replace\",\"pattern\":{\"String\":123},...} or {\"String\":null} — a 'String' member present with a non-string value.","commonSituations":"Generating tokenizer.json from code that didn't quote the value; YAML-to-JSON conversion dropping quotes; hand edits.","solutions":["Quote the value: \"pattern\": {\"String\": \"literal text\"}.","If you intended a regex, use the \"Regex\" member instead of \"String\".","Validate tokenizer.json with the Python tokenizers library before using it in .NET."],"exampleFix":"// before\n\"pattern\": {\"String\": 42}\n// after\n\"pattern\": {\"String\": \"42\"}","handlingStrategy":"validation","validationCode":"if (pattern.TryGetProperty(\"String\", out var s) && s.ValueKind != JsonValueKind.String)\n    throw new Exception(\"Replace 'String' pattern must be a JSON string\");","typeGuard":"bool IsStringPattern(JsonElement pattern) => pattern.TryGetProperty(\"String\", out var s) && s.ValueKind == JsonValueKind.String;","tryCatchPattern":"try { LoadTokenizer(path); }\ncatch (InvalidDataException ex) when (ex.Message.Contains(\"'String' pattern must be a string\")) {\n    // fix the JSON member type and retry\n}","preventionTips":["Serialize literal patterns with explicit string quoting","Run JSON schema validation on tokenizer.json in CI","Beware YAML/TS generators that emit unquoted values"],"tags":["tokenizer","normalizer","type-mismatch","json"],"backgroundTag":"type-mismatch","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}