{"record":{"id":"9397fb8149f41706","repo":"dotnet/machinelearning","slug":"argumentnullexception-9397fb","errorCode":null,"errorMessage":"ArgumentNullException","messagePattern":"ArgumentNullException","errorType":"exception","errorClass":"ArgumentNullException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.Tokenizers/Model/SentencePieceBaseModel.cs","lineNumber":22,"sourceCode":"\nusing Sentencepiece;\nusing System;\nusing System.Buffers;\nusing System.Collections.Generic;\nusing System.Diagnostics;\nusing System.Linq;\nusing System.Text;\nusing System.Text.RegularExpressions;\n\nnamespace Microsoft.ML.Tokenizers\n{\n    internal abstract class SentencePieceBaseModel\n    {\n        internal SentencePieceBaseModel(ModelProto modelProto, bool addBos = false, bool addEos = false, IReadOnlyDictionary<string, int>? specialTokens = null)\n        {\n            if (modelProto is null)\n            {\n                throw new ArgumentNullException(nameof(modelProto));\n            }\n\n            AddBeginningOfSentence = addBos;\n            AddEndOfSentence = addEos;\n            BeginningOfSentenceToken = modelProto.TrainerSpec.BosPiece ?? \"<s>\";\n            BeginningOfSentenceId = Math.Max(0, modelProto.TrainerSpec.BosId);\n            EndOfSentenceToken = modelProto.TrainerSpec.EosPiece ?? \"</s>\";\n            EndOfSentenceId = Math.Max(0, modelProto.TrainerSpec.EosId);\n            UnknownToken = modelProto.TrainerSpec.UnkPiece ?? \"<unk>\";\n            UnknownId = Math.Max(0, modelProto.TrainerSpec.UnkId);\n            AddDummyPrefix = modelProto.NormalizerSpec.AddDummyPrefix;\n            EscapeWhiteSpaces = modelProto.NormalizerSpec.EscapeWhitespaces;\n            TreatWhitespaceAsSuffix = modelProto.TrainerSpec.TreatWhitespaceAsSuffix;\n            ByteFallback = modelProto.TrainerSpec.ByteFallback;\n            SpecialTokens = specialTokens;\n\n            if (specialTokens is not null && specialTokens.Count > 0)\n            {","sourceCodeStart":4,"sourceCodeEnd":40,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.Tokenizers/Model/SentencePieceBaseModel.cs#L4-L40","documentation":"The SentencePieceBaseModel constructor requires a non-null SentencePiece ModelProto; this guard throws ArgumentNullException when modelProto is null. It is the base initializer for Llama/Phi SentencePiece tokenizers.","triggerScenarios":"Constructing a derived SentencePiece model (e.g. via LlamaTokenizer.Create) with a ModelProto variable that is null because parsing the .model file failed or returned null.","commonSituations":"ModelProto.Parser.ParseFrom silently handling empty byte arrays downstream; a nullable proto passed along after a failed load; refactored caching returning null.","solutions":["Ensure the .model file was read and parsed successfully before constructing the tokenizer","Null-check the parsed ModelProto before passing it in","Throw or log a clear error at load time instead of propagating null"],"exampleFix":"// before\nModelProto? proto = TryLoadModel(path); // may be null\nvar tok = LlamaTokenizer.Create(proto);\n// after\nvar proto = ModelProto.Parser.ParseFrom(File.ReadAllBytes(path));\nvar tok = LlamaTokenizer.Create(proto);","handlingStrategy":"validation","validationCode":"if (modelProto is null) throw new InvalidOperationException(\"ModelProto must be parsed before constructing the tokenizer.\");","typeGuard":null,"tryCatchPattern":"try { var tok = LlamaTokenizer.Create(modelProto); } catch (ArgumentNullException ex) { log.LogError(ex, \"modelProto was null\"); throw; }","preventionTips":["Parse the .model file eagerly and fail fast on load errors","Avoid nullable ModelProto variables crossing API boundaries","Centralize model loading with null assertions"],"tags":["tokenizers","sentencepiece","null-check"],"backgroundTag":"null-argument","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}