{"record":{"id":"0118cbf8977cdc36","repo":"dotnet/machinelearning","slug":"the-max-token-count-must-be-greater-than-0-0118cb","errorCode":null,"errorMessage":"The max token count must be greater than 0.","messagePattern":"The max token count must be greater than 0\\.","errorType":"exception","errorClass":"ArgumentOutOfRangeException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.Tokenizers/Model/EnglishRobertaTokenizer.cs","lineNumber":546,"sourceCode":"                    if (length < split.Length || count >= maxTokenCount)\n                    {\n                        break;\n                    }\n                }\n            }\n            else\n            {\n                count += EncodeToIdsInternal(textSpanToEncode, null, out charsConsumed, maxTokenCount);\n            }\n\n            return count;\n        }\n\n        private int LastIndexOf(string? text, ReadOnlySpan<char> textSpan, int maxTokenCount, bool considerPreTokenization, bool considerNormalization, out string? normalizedText, out int tokenCount)\n        {\n            if (maxTokenCount <= 0)\n            {\n                throw new ArgumentOutOfRangeException(nameof(maxTokenCount), \"The max token count must be greater than 0.\");\n            }\n\n            if (string.IsNullOrEmpty(text) && textSpan.IsEmpty)\n            {\n                normalizedText = null;\n                tokenCount = 0;\n                return 0;\n            }\n\n            IEnumerable<(int Offset, int Length)>? splits = InitializeForEncoding(\n                                                                text,\n                                                                textSpan,\n                                                                considerPreTokenization,\n                                                                considerNormalization,\n                                                                _normalizer,\n                                                                _preTokenizer,\n                                                                out normalizedText,\n                                                                out ReadOnlySpan<char> textSpanToEncode,","sourceCodeStart":528,"sourceCodeEnd":564,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.Tokenizers/Model/EnglishRobertaTokenizer.cs#L528-L564","documentation":"LastIndexOf (used by GetIndexByTokenCount when searching backwards) requires maxTokenCount > 0 and throws ArgumentOutOfRangeException otherwise. Note the message wording differs slightly from the EncodeToIds/CountTokens validation ('must be greater than 0').","triggerScenarios":"Calling GetIndexByTokenCount with a text-splitting/token-count value of 0 or negative, which is forwarded to LastIndexOf as maxTokenCount.","commonSituations":"Computing the 'last N tokens' N from a remainder calculation that yielded 0; config value of 0 meant as 'disable' but the API has no such semantics; off-by-one in a loop decrementing the count.","solutions":["Ensure the token count passed is at least 1 before calling GetIndexByTokenCount.","Handle the zero case at the call site (there is nothing to search when the limit is 0) instead of calling the API.","Clamp: max = Math.Max(1, requestedCount)."],"exampleFix":"// before\nint idx = tokenizer.GetIndexByTokenCount(text, false, false, out _, maxTokenCount: remaining); // remaining could be 0\n// after\nif (remaining <= 0) return -1; // nothing to search\nint idx = tokenizer.GetIndexByTokenCount(text, false, false, out _, maxTokenCount: remaining);","handlingStrategy":"validation","validationCode":"if (maxTokenCount <= 0)\n{\n    if (maxTokenCount == 0) return -1; // nothing to search\n    throw new InvalidOperationException(\"maxTokenCount must be >= 1\");\n}","typeGuard":null,"tryCatchPattern":"try { int idx = tokenizer.GetIndexByTokenCount(text, false, false, out _, maxTokenCount: n); }\ncatch (ArgumentOutOfRangeException ex) when (ex.ParamName == \"maxTokenCount\") { throw new InvalidOperationException(\"Backward token count must be positive\", ex); }","preventionTips":["Handle limit == 0 at the call site (empty result) instead of calling the API.","Avoid off-by-one arithmetic when computing the 'last N tokens'.","Map config values like 0 ('disabled') to explicit semantics before passing through.","Clamp with Math.Max(1, value) when a minimum of one token is required."],"tags":["argument-validation","tokenizer","csharp"],"backgroundTag":"argument-out-of-range","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}