{"record":{"id":"e2fd2d25b6016191","repo":"dotnet/machinelearning","slug":"invalid-json","errorCode":null,"errorMessage":"Invalid JSON.","messagePattern":"Invalid JSON\\.","errorType":"exception","errorClass":"JsonException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.Tokenizers/Utils/StringSpanOrdinalKey.cs","lineNumber":166,"sourceCode":"        public override Vocabulary Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)\n        {\n            var dictionary = new Vocabulary();\n            while (reader.Read())\n            {\n                if (reader.TokenType == JsonTokenType.EndObject)\n                {\n                    return dictionary;\n                }\n\n                if (reader.TokenType == JsonTokenType.PropertyName)\n                {\n                    var key = reader.GetString();\n                    reader.Read();\n                    var value = reader.GetInt32();\n                    dictionary.Add(new StringSpanOrdinalKey(key!), (value, key!));\n                }\n            }\n            throw new JsonException(\"Invalid JSON.\");\n        }\n\n        public override void Write(Utf8JsonWriter writer, Vocabulary value, JsonSerializerOptions options) => throw new NotImplementedException();\n    }\n\n    /// <summary>\n    /// Extension methods for <see cref=\"StringSpanOrdinalKey\"/>.\n    /// </summary>\n    internal static class StringSpanOrdinalKeyExtensions\n    {\n        public static unsafe bool TryGetValue<TValue>(this Dictionary<StringSpanOrdinalKey, TValue> map, ReadOnlySpan<char> key, out TValue value)\n        {\n            fixed (char* ptr = key)\n            {\n                return map.TryGetValue(new StringSpanOrdinalKey(ptr, key.Length), out value!);\n            }\n        }\n","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.Tokenizers/Utils/StringSpanOrdinalKey.cs#L148-L184","documentation":"This JsonException is the fall-through failure of a custom Vocabulary JsonConverter's Read method: it reads a JSON object of string->int mappings and throws 'Invalid JSON.' whenever the structure doesn't match (non-object token, non-integer value, missing properties, or the reader never reaching EndObject). It exists to give a consistent failure when the serialized vocabulary doesn't match the expected shape.","triggerScenarios":"Deserializing a Vocabulary from JSON that is not a flat object of string keys to integer values — e.g. an array instead of an object, nested values, string values for IDs, or truncated JSON.","commonSituations":"Loading a vocab.json saved in a different format/version than the converter expects, hand-editing the vocab file and breaking the schema, or passing a non-vocab JSON stream to the deserializer.","solutions":["Check the JSON input is a flat {\"token\": id, ...} object with integer values; fix or regenerate the file.","Compare against the format produced by the same library version's Write/serialization path.","Deserialize from the original vocab source rather than a hand-edited copy.","Catch JsonException and inspect reader.TokenType/position to pinpoint the offending JSON location."],"exampleFix":"// before: values as strings\n{ \"hello\": \"12\", \"world\": \"34\" }\n// after\n{ \"hello\": 12, \"world\": 34 }","handlingStrategy":"try-catch","validationCode":"using var doc = JsonDocument.Parse(json);\nbool validShape = doc.RootElement.ValueKind == JsonValueKind.Object && doc.RootElement.EnumerateObject().All(p => p.Value.ValueKind == JsonValueKind.Number);","typeGuard":"static bool LooksLikeVocabulary(string json) { try { using var d = JsonDocument.Parse(json); return d.RootElement.ValueKind == JsonValueKind.Object; } catch (JsonException) { return false; } }","tryCatchPattern":"try { vocab = JsonSerializer.Deserialize<Vocabulary>(json, options); }\ncatch (JsonException ex)\n{ throw new InvalidDataException(\"Vocabulary JSON is not a string->int object\", ex); }","preventionTips":["Verify vocab JSON is a flat object with integer values before deserializing","Regenerate vocab files with the same library version doing the deserialization","Avoid hand-editing serialized vocab files"],"tags":["json","deserialization","vocabulary","converter"],"backgroundTag":"json-parse-error","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}