{"record":{"id":"22a66bfd22fbe904","repo":"dotnet/machinelearning","slug":"argumentnullexception","errorCode":null,"errorMessage":"ArgumentNullException","messagePattern":"ArgumentNullException","errorType":"exception","errorClass":"ArgumentNullException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.Tokenizers/Model/Phi2Tokenizer.cs","lineNumber":108,"sourceCode":"        /// <returns>The CodeGen tokenizer object.</returns>\n        /// <remarks>\n        /// The tokenizer will be created according to the configuration specified in https://huggingface.co/microsoft/phi-2/raw/main/tokenizer.json.\n        /// It is important to provide the similar vocab and merges files to the ones used in the training of the model.\n        /// The vocab and merges files can be downloaded from the following links:\n        ///     https://huggingface.co/microsoft/phi-2/resolve/main/vocab.json?download=true\n        ///     https://huggingface.co/microsoft/phi-2/resolve/main/merges.txt?download=true\n        /// When creating the tokenizer, ensure that the vocabulary stream is sourced from a trusted provider.\n        /// </remarks>\n        public static new Phi2Tokenizer Create(\n            Stream vocabStream,\n            Stream mergesStream,\n            bool addPrefixSpace = false,\n            bool addBeginOfSentence = false,\n            bool addEndOfSentence = false)\n        {\n            if (vocabStream is null)\n            {\n                throw new ArgumentNullException(nameof(vocabStream));\n            }\n\n            if (mergesStream is null)\n            {\n                throw new ArgumentNullException(nameof(mergesStream));\n            }\n\n            return new Phi2Tokenizer(\n                        vocabStream, mergesStream, new RegexPreTokenizer(TiktokenTokenizer.P50kBaseRegex(), CodeGenTokenizer.CodeGenSpecialTokens), normalizer: null,\n                        CodeGenTokenizer.CodeGenSpecialTokens, addPrefixSpace: addPrefixSpace, addBeginningOfSentence: addBeginOfSentence, addEndOfSentence: addEndOfSentence);\n        }\n    }\n}\n","sourceCodeStart":90,"sourceCodeEnd":122,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.Tokenizers/Model/Phi2Tokenizer.cs#L90-L122","documentation":"Null-argument guard inside Phi2Tokenizer.Create: while building the CodeGen/Phi-2 tokenizer from the vocab, merges and configuration streams, one of the required arguments (the stream/JSON configuration or vocabulary source) is null, and an ArgumentNullException is thrown. The Create method refuses to proceed because the tokenizer's BPE tables cannot be loaded from nothing.","triggerScenarios":"Calling Phi2Tokenizer.Create(vocabStream: null, mergesStream, ...) or passing a variable that was never assigned because the vocab resource failed to load.","commonSituations":"Assembly resource name typo so GetManifestResourceStream returns null; file not found handled by returning null stream; refactor renamed a variable.","solutions":["Pass a valid non-null Stream for the vocab file (e.g. File.OpenRead(\"vocab.json\"))","Check that the embedded resource name is correct when using GetManifestResourceStream","Null-check streams before calling Create and surface a clear load error"],"exampleFix":"// before\nStream? vocab = assembly.GetManifestResourceStream(\"Wrong.Name.vocab.json\"); // null\nvar tok = Phi2Tokenizer.Create(vocab, merges);\n// after\nusing Stream vocab = File.OpenRead(\"vocab.json\");\nvar tok = Phi2Tokenizer.Create(vocab, merges);","handlingStrategy":"validation","validationCode":"if (vocabStream is null) throw new InvalidOperationException(\"Vocab stream failed to load; check the resource name/path.\");","typeGuard":null,"tryCatchPattern":"try { var tok = Phi2Tokenizer.Create(vocabStream, mergesStream); } catch (ArgumentNullException ex) { log.LogError(ex, \"vocabStream was null\"); throw; }","preventionTips":["Check GetManifestResourceStream results for null immediately","Use File.OpenRead and let missing files throw loudly","Null-check model asset streams before tokenizer construction"],"tags":["tokenizers","null-check","argument-null"],"backgroundTag":"null-argument","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}