dotnet/machinelearning · error · ArgumentNullException

throw new ArgumentNullException(nameof(ids));

Error message

throw new ArgumentNullException(nameof(ids));

What it means

The string-returning Decode overload requires a non-null sequence of token IDs. A null ids argument is rejected immediately with ArgumentNullException because there is no meaningful decode target. Empty sequences are allowed and decode to an empty string.

Source

Thrown at src/Microsoft.ML.Tokenizers/Model/BPETokenizer.cs:785

        /// <summary>
        /// Decode the given ids, back to a String.
        /// </summary>
        /// <param name="ids">The list of ids that we want to decode.</param>
        /// <returns>The decoded string.</returns>
        public override string Decode(IEnumerable<int> ids) => Decode(ids, considerSpecialTokens: true);

        /// <summary>
        /// Decode the given ids, back to a String.
        /// </summary>
        /// <param name="ids">The list of ids that we want to decode.</param>
        /// <param name="considerSpecialTokens">Indicate whether to consider special tokens or not.</param>
        /// <returns>The decoded string.</returns>
        public string Decode(IEnumerable<int> ids, bool considerSpecialTokens)
        {
            if (ids is null)
            {
                throw new ArgumentNullException(nameof(ids));
            }

            if (ByteLevel)
            {
                return DecodeByteLevel(ids, considerSpecialTokens);
            }

            ValueStringBuilder sb = new ValueStringBuilder();

            foreach (int id in ids)
            {
                if (_specialTokensReverse?.TryGetValue(id, out string? token) is true)
                {
                    if (considerSpecialTokens)
                    {
                        sb.Append(token);
                    }
                    continue;

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Ensure the IDs collection is initialized (e.g. Array.Empty<int>()) before decoding.
2. Check the source of the IDs for a null-producing error path and handle it before decoding.
3. Coalesce null to an empty sequence if decoding nothing is acceptable.

Example fix

// before
string text = tokenizer.Decode(ids, considerSpecialTokens: false); // ids is null
// after
string text = tokenizer.Decode(ids ?? (IEnumerable<int>)Array.Empty<int>(), considerSpecialTokens: false);

if (ids is null) throw new InvalidOperationException("No token IDs to decode; encoding step failed.");

bool HasIds(IEnumerable<int>? ids) => ids is not null;

try { text = tokenizer.Decode(ids, false); }
catch (ArgumentNullException ex) when (ex.ParamName == "ids") { text = string.Empty; /* surface upstream failure */ }

When it happens

Trigger: Calling BpeTokenizer.Decode(IEnumerable<int> ids, bool considerSpecialTokens) with ids == null, typically when the variable holding generated token IDs was never assigned or a decode path ran after a failed encode returned null.

Common situations: Pipelines where model output IDs are null on error but the code decodes unconditionally; deserialized objects with a null Ids property.

Related errors

• Value cannot be null. (Parameter 'input')
• columns
• rawType
• Invalid input stream contents
• Invalid path

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/01212530ada7c273. Report an issue: GitHub.