{"record":{"id":"53156647d245b5be","repo":"dotnet/machinelearning","slug":"failed-to-insert-key-negative-value","errorCode":null,"errorMessage":"failed to insert key: negative value","messagePattern":"failed to insert key: negative value","errorType":"validation","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.Tokenizers/Utils/DoubleArrayTrie.cs","lineNumber":342,"sourceCode":"            return key;\n        }\n\n        private void FreeNode(uint id) => _recycleBin.Push(id);\n\n        public void Finish()\n        {\n            Flush(0);\n\n            _units[0] = _nodes[0].Unit;\n            _labels[0] = _nodes[0].Label;\n            _isIntersections.Build();\n        }\n\n        public void Insert(ReadOnlySpan<byte> key, int length, int value)\n        {\n            if (value < 0)\n            {\n                throw new ArgumentException(\"failed to insert key: negative value\");\n            }\n            else if (length == 0)\n            {\n                throw new ArgumentException(\"failed to insert key: zero-length key\");\n            }\n\n            uint id = 0;\n            int keyPos = 0;\n\n            for (; keyPos <= length; ++keyPos)\n            {\n                uint childId = _nodes[(int)id].Child;\n                if (childId == 0)\n                {\n                    break;\n                }\n\n                byte keyLabel = key[keyPos];","sourceCodeStart":324,"sourceCodeEnd":360,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.Tokenizers/Utils/DoubleArrayTrie.cs#L324-L360","documentation":"DoubleArrayTrie.Insert throws this ArgumentException when asked to insert a key with a negative associated value. The trie stores key values in fields that are interpreted as unsigned/ID-like data, so negative values are meaningless and would corrupt the structure. It is a fail-fast guard used when building the trie (e.g. from BuildDawg during vocabulary loading).","triggerScenarios":"Calling trie.Insert(spanKey, length, value) with a negative value, which happens if a vocabulary build assigns negative token IDs/indices (e.g. from an unchecked subtraction or an uninitialized field).","commonSituations":"Building a DoubleArrayTrie from a sentencepiece vocab where a token's score/index computation yields a negative number, or a custom caller inserting arbitrary values directly.","solutions":["Fix the value source so keys are inserted with non-negative values (clamp or validate before Insert).","Check the code computing the value (e.g. token index assignment) for off-by-one or subtraction errors.","If negative values are semantically needed, store an offset/shifted value instead.","Assert value >= 0 at the collection construction site to catch it earlier."],"exampleFix":"// before\ntrie.Insert(keyBytes, keyLength, tokenIndex - 1);\n// after\nint value = tokenIndex - 1;\nif (value < 0) throw new ArgumentException($\"Token index must be non-negative, got {value}\");\ntrie.Insert(keyBytes, keyLength, value);","handlingStrategy":"validation","validationCode":"if (value < 0) throw new ArgumentException($\"Trie value must be non-negative, got {value}\");\ntrie.Insert(key, length, value);","typeGuard":"static bool IsValidTrieEntry(int value, int length) => value >= 0 && length > 0;","tryCatchPattern":"try { trie.Insert(key, length, value); }\ncatch (ArgumentException ex) when (ex.Message.Contains(\"negative value\"))\n{ /* fix value source or skip entry */ }","preventionTips":["Validate value >= 0 at the vocabulary-build site","Check index computations for negative underflow","Use uint for ID-like values upstream"],"tags":["trie","argument","validation","tokenizer"],"backgroundTag":"invalid-argument-value","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}