dotnet/machinelearning · error · ArgumentNullException

throw new ArgumentNullException(nameof(vocabulary));

Error message

throw new ArgumentNullException(nameof(vocabulary));

What it means

The BpeOptions constructor throws ArgumentNullException when the vocabulary parameter is null. BpeOptions requires an IEnumerable<KeyValuePair<string,int>> mapping tokens/merges to IDs, and a BPE tokenizer cannot operate without one. The docs explicitly document this exception.

Source

Thrown at src/Microsoft.ML.Tokenizers/Model/BpeOptions.cs:26

using System.Text.Json;

namespace Microsoft.ML.Tokenizers
{
    /// <summary>
    /// Options for the BPE tokenizer.
    /// </summary>
    public sealed class BpeOptions
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="BpeOptions"/> class.
        /// </summary>
        /// <param name="vocabulary">The vocabulary to use.</param>
        /// <exception cref="ArgumentNullException">Thrown when <paramref name="vocabulary"/> is null.</exception>
        public BpeOptions(IEnumerable<KeyValuePair<string, int>> vocabulary)
        {
            if (vocabulary == null)
            {
                throw new ArgumentNullException(nameof(vocabulary));
            }

            Vocabulary = vocabulary;
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="BpeOptions"/> class.
        /// </summary>
        /// <param name="vocabFile">The JSON file path containing the dictionary of string keys and their ids.</param>
        /// <param name="mergesFile">The file path containing the tokens's pairs list.</param>
        public BpeOptions(string vocabFile, string? mergesFile = null)
        {
            if (vocabFile is null)
            {
                throw new ArgumentNullException(nameof(vocabFile));
            }

            if (!File.Exists(vocabFile))

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Pass a non-null vocabulary dictionary of token-to-ID pairs.
2. Validate the vocab load step; make loaders throw on failure instead of returning null.
3. Use BpeTokenizer.Create(file paths) overloads that load the vocab for you.
4. Fail fast at startup if the vocabulary is missing rather than at tokenizer construction.

Example fix

// before
var vocab = LoadVocab(); // may return null
var options = new BpeOptions(vocab);
// after
var vocab = LoadVocab() ?? throw new InvalidOperationException("BPE vocabulary failed to load");
var options = new BpeOptions(vocab);

if (vocabulary is null) throw new InvalidOperationException("BPE vocabulary must be loaded before constructing BpeOptions");
if (!vocabulary.Any()) throw new InvalidOperationException("BPE vocabulary must not be empty");

static bool HasVocabulary(IEnumerable<KeyValuePair<string, int>>? v) => v is not null && v.Any();

try { var opts = new BpeOptions(vocabulary); } catch (ArgumentNullException ex) when (ex.ParamName == "vocabulary") { /* load vocab from disk and retry */ }

When it happens

Trigger: new BpeOptions(null) — typically when the vocabulary is loaded lazily and the load returned null, or a config-driven dictionary failed to populate before construction.

Common situations: BPE vocab JSON/merge files that failed to load silently; factory methods returning null vocabularies on missing files; DI scenarios where the vocab provider wasn't registered.

Related errors

• throw new ArgumentNullException(nameof(vocabStream));
• throw new ArgumentNullException(nameof(vocabFilePath));
• throw new ArgumentNullException(nameof(vocabStream));
• throw new ArgumentNullException(nameof(vocabFile));
• throw new ArgumentNullException(nameof(vocabularyPath));

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/67cfcdfa695166fc. Report an issue: GitHub.