{"record":{"id":"c72d67754e00346c","repo":"dotnet/machinelearning","slug":"unknown-or-unsupported-protobuf-wire-type-wiretyp","errorCode":null,"errorMessage":"Unknown or unsupported protobuf wire type {wireType}.","messagePattern":"Unknown or unsupported protobuf wire type (.+?)\\.","errorType":"validation","errorClass":"InvalidDataException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.Tokenizers/SentencepieceModel.cs","lineNumber":158,"sourceCode":"                    }\n                    pos += 8;\n                    break;\n\n                case 2: // length-delimited\n                    int skipLength = ReadLengthPrefix(data, end, ref pos);\n                    pos += skipLength;\n                    break;\n\n                case 5: // 32-bit fixed\n                    if (pos > end - 4)\n                    {\n                        throw new InvalidDataException(\"Unexpected end of data while skipping fixed32.\");\n                    }\n                    pos += 4;\n                    break;\n\n                default:\n                    throw new InvalidDataException($\"Unknown or unsupported protobuf wire type {wireType}.\");\n            }\n        }\n    }\n\n    /// <summary>Lightweight replacement for Google.Protobuf.ByteString with a Span property.</summary>\n    internal readonly struct SentencePieceByteString(byte[] data, int offset, int length)\n    {\n        internal ReadOnlySpan<byte> Span => data is null ? ReadOnlySpan<byte>.Empty : data.AsSpan(offset, length);\n    }\n\n    /// <summary>ModelProto  (top-level message; field numbers match sentencepiece_model.proto)</summary>\n    internal sealed class ModelProto\n    {\n        internal static readonly ModelProtoParser Parser = new();\n\n        internal List<Types.SentencePiece> Pieces { get; } = new();\n        internal TrainerSpec TrainerSpec { get; private set; } = new();\n        internal NormalizerSpec NormalizerSpec { get; private set; } = new();","sourceCodeStart":140,"sourceCodeEnd":176,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.Tokenizers/SentencepieceModel.cs#L140-L176","documentation":"This InvalidDataException is thrown by the hand-written protobuf SkipField helper in SentencepieceModel.cs when it encounters a wire type it does not know how to skip while reading a sentencepiece model protobuf. The library only supports the standard wire types (varint, fixed64, length-delimited, fixed32); anything else means the stream is corrupt or not actually a protobuf message. It acts as a guard against silently consuming a malformed model file.","triggerScenarios":"Deserializing a sentencepiece .model file whose encoded tag byte contains a wire type other than 0, 1, 2, or 5 — e.g. the file is truncated and misaligned so a value byte is parsed as a tag, or the file is not a protobuf at all (HTML error page, wrong file downloaded).","commonSituations":"Downloading a corrupted or partial tokenizer model, pointing the tokenizer at the wrong file, a model serialized by an incompatible/newer protobuf schema using newer wire types (e.g. wire type 3/4 groups or 6+), or manually concatenating protobuf streams.","solutions":["Re-download or regenerate the sentencepiece .model file and verify its integrity (checksum/size).","Confirm the file being passed is actually a protobuf sentencepiece model (open it — it should start as compact binary, typically beginning with byte 0x0A for field 1).","Check that no preprocessing step (transfer mode, encoding conversion, line-ending munging) is corrupting the file before parsing.","If building from source text, regenerate with the same sentencepiece trainer version used by the library."],"exampleFix":"// before: pointing at a wrong/HTML file downloaded from a URL\nvar tokenizer = SentencePieceTokenizer.Create(modelPath: \"model.html\");\n// after: validate the file is a protobuf before loading\nbyte[] head = File.ReadAllBytes(\"model.html\")[..2];\nif (head[0] != 0x0A) throw new InvalidDataException(\"Not a sentencepiece protobuf model\");\nvar tokenizer = SentencePieceTokenizer.Create(modelPath: \"model.html\");","handlingStrategy":"try-catch","validationCode":"byte[] head = new byte[2];\nusing (var fs = File.OpenRead(modelPath)) { if (fs.Read(head) < 2) throw new InvalidDataException(\"Model file too small\"); }\nbool likelyProtobuf = head[0] == 0x0A;","typeGuard":"static bool IsLikelySentencepieceModel(byte[] data) => data is { Length: > 2 } && data[0] == 0x0A && data[1] > 0;","tryCatchPattern":"try { tokenizer = SentencePieceTokenizer.Create(modelStream); }\ncatch (InvalidDataException ex) when (ex.Message.Contains(\"wire type\"))\n{ throw new ApplicationException(\"Model file is corrupt or not a protobuf sentencepiece model\", ex); }","preventionTips":["Verify model file checksums after download","Never open the model file in text/transfer modes that alter bytes","Pin the model file source and validate the first byte is 0x0A before loading"],"tags":["protobuf","tokenizer","corrupt-file","deserialization"],"backgroundTag":"protobuf-unmarshal-failed","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}