{"record":{"id":"11fadecf99448caf","repo":"dotnet/machinelearning","slug":"throw-new-argumentnullexception-nameof-tokenids","errorCode":null,"errorMessage":"throw new ArgumentNullException(nameof(tokenIds));","messagePattern":"throw new ArgumentNullException\\(nameof\\(tokenIds\\)\\);","errorType":"exception","errorClass":"ArgumentNullException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.Tokenizers/Model/BertTokenizer.cs","lineNumber":294,"sourceCode":"            }\n\n            return ids;\n        }\n\n        /// <summary>\n        /// Build model inputs from a sequence or a pair of sequence for sequence classification tasks by concatenating and adding special tokens. A BERT sequence has the following format:\n        ///     - single sequence: `[CLS] tokenIds [SEP]`\n        ///     - pair of sequences: `[CLS] tokenIds [SEP] additionalTokenIds [SEP]`\n        /// </summary>\n        /// <param name=\"tokenIds\">List of IDs to which the special tokens will be added.</param>\n        /// <param name=\"additionalTokenIds\">Optional second list of IDs for sequence pairs.</param>\n        /// <returns>The list of IDs with special tokens added.</returns>\n        /// <exception cref=\"ArgumentNullException\">When <paramref name=\"tokenIds\"/> is null.</exception>\n        public IReadOnlyList<int> BuildInputsWithSpecialTokens(IEnumerable<int> tokenIds, IEnumerable<int>? additionalTokenIds = null)\n        {\n            if (tokenIds is null)\n            {\n                throw new ArgumentNullException(nameof(tokenIds));\n            }\n\n            List<int> ids;\n\n            if (tokenIds is ICollection<int> c1)\n            {\n                int capacity = c1.Count + 2;    // Add 2 for [CLS] and two [SEP] tokens.\n\n                if (additionalTokenIds is not null)\n                {\n                    capacity += additionalTokenIds is ICollection<int> c2 ? c2.Count + 1 : c1.Count + 1;\n                }\n\n                ids = new(capacity) { ClassificationTokenId };\n            }\n            else\n            {\n                // slow path","sourceCodeStart":276,"sourceCodeEnd":312,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.Tokenizers/Model/BertTokenizer.cs#L276-L312","documentation":"BertTokenizer.BuildInputsWithSpecialTokens (list overload) rejects a null tokenIds argument with ArgumentNullException, since special tokens (CLS/SEP) cannot be wrapped around an absent sequence. The method documents this exception in its XML docs.","triggerScenarios":"Calling BuildInputsWithSpecialTokens(IEnumerable<int> tokenIds, ...) with tokenIds == null, e.g. when an upstream encode step failed and returned null instead of a list.","commonSituations":"Preparing BERT model inputs where tokenization failures propagate null; deserialized request payloads missing the token IDs.","solutions":["Initialize tokenIds to an empty collection before the call.","Handle the upstream null/error before building inputs.","Check the encode result for null before passing it on."],"exampleFix":"// before\nvar inputs = tokenizer.BuildInputsWithSpecialTokens(tokenIds); // tokenIds null after failed encode\n// after\nif (tokenIds is null) throw new InvalidOperationException(\"tokenization failed\");\nvar inputs = tokenizer.BuildInputsWithSpecialTokens(tokenIds);","handlingStrategy":"validation","validationCode":"if (tokenIds is null) throw new InvalidOperationException(\"tokenIds must be produced before building BERT inputs\");","typeGuard":"bool HasTokenIds(IEnumerable<int>? ids) => ids is not null;","tryCatchPattern":"try { inputs = tokenizer.BuildInputsWithSpecialTokens(tokenIds); }\ncatch (ArgumentNullException ex) when (ex.ParamName == \"tokenIds\") { /* handle failed tokenization upstream */ }","preventionTips":["Check the result of EncodeToIds for null before building inputs.","Use non-nullable collection types in your pipeline.","Default missing IDs to empty lists."],"tags":["csharp","null-argument","bert"],"backgroundTag":"null-argument","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}