dotnet/machinelearning · error · ArgumentException

failed to insert key: negative value

Error message

failed to insert key: negative value

What it means

DoubleArrayTrie.Insert throws this ArgumentException when asked to insert a key with a negative associated value. The trie stores key values in fields that are interpreted as unsigned/ID-like data, so negative values are meaningless and would corrupt the structure. It is a fail-fast guard used when building the trie (e.g. from BuildDawg during vocabulary loading).

Source

Thrown at src/Microsoft.ML.Tokenizers/Utils/DoubleArrayTrie.cs:342

            return key;
        }

        private void FreeNode(uint id) => _recycleBin.Push(id);

        public void Finish()
        {
            Flush(0);

            _units[0] = _nodes[0].Unit;
            _labels[0] = _nodes[0].Label;
            _isIntersections.Build();
        }

        public void Insert(ReadOnlySpan<byte> key, int length, int value)
        {
            if (value < 0)
            {
                throw new ArgumentException("failed to insert key: negative value");
            }
            else if (length == 0)
            {
                throw new ArgumentException("failed to insert key: zero-length key");
            }

            uint id = 0;
            int keyPos = 0;

            for (; keyPos <= length; ++keyPos)
            {
                uint childId = _nodes[(int)id].Child;
                if (childId == 0)
                {
                    break;
                }

                byte keyLabel = key[keyPos];

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Fix the value source so keys are inserted with non-negative values (clamp or validate before Insert).
2. Check the code computing the value (e.g. token index assignment) for off-by-one or subtraction errors.
3. If negative values are semantically needed, store an offset/shifted value instead.
4. Assert value >= 0 at the collection construction site to catch it earlier.

Example fix

// before
trie.Insert(keyBytes, keyLength, tokenIndex - 1);
// after
int value = tokenIndex - 1;
if (value < 0) throw new ArgumentException($"Token index must be non-negative, got {value}");
trie.Insert(keyBytes, keyLength, value);

if (value < 0) throw new ArgumentException($"Trie value must be non-negative, got {value}");
trie.Insert(key, length, value);

static bool IsValidTrieEntry(int value, int length) => value >= 0 && length > 0;

try { trie.Insert(key, length, value); }
catch (ArgumentException ex) when (ex.Message.Contains("negative value"))
{ /* fix value source or skip entry */ }

When it happens

Trigger: Calling trie.Insert(spanKey, length, value) with a negative value, which happens if a vocabulary build assigns negative token IDs/indices (e.g. from an unchecked subtraction or an uninitialized field).

Common situations: Building a DoubleArrayTrie from a sentencepiece vocab where a token's score/index computation yields a negative number, or a custom caller inserting arbitrary values directly.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• failed to insert key: zero-length key
• Expected either {0} or {1} to be provided
• Blob for normalization rule is broken.
• Too many pre-tokenizers provided. Maximum is {MaxPreTokenize
• failed to insert key: invalid null character

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/53156647d245b5be. Report an issue: GitHub.