dotnet/machinelearning · error · ArgumentNullException

throw new ArgumentNullException(nameof(vocabStream));

Error message

throw new ArgumentNullException(nameof(vocabStream));

What it means

The async BertTokenizer.CreateAsync(Stream vocabStream, ...) throws ArgumentNullException when the supplied vocabStream is null. The stream is validated before LoadVocabAsync reads the vocabulary from it, because there is nothing to load from a null stream.

Source

Thrown at src/Microsoft.ML.Tokenizers/Model/BertTokenizer.cs:699

        /// <summary>
        /// Create a new instance of the <see cref="BertTokenizer"/> class asynchronously.
        /// </summary>
        /// <param name="vocabStream">The stream containing the vocabulary file.</param>
        /// <param name="options">The options to use for the Bert tokenizer.</param>
        /// <param name="cancellationToken">The cancellation token.</param>
        /// <returns>A task that represents the asynchronous creation of the BertTokenizer.</returns>
        /// <remarks>
        /// When creating the tokenizer, ensure that the vocabulary stream is sourced from a trusted provider.
        /// </remarks>
        public static async Task<BertTokenizer> CreateAsync(
                    Stream vocabStream,
                    BertOptions? options = null,
                    CancellationToken cancellationToken = default)
        {
            if (vocabStream is null)
            {
                throw new ArgumentNullException(nameof(vocabStream));
            }

            (Dictionary<StringSpanOrdinalKey, int> vocab, Dictionary<int, string> vocabReverse) = await LoadVocabAsync(vocabStream, useAsync: true, cancellationToken).ConfigureAwait(false);

            return Create(vocab, vocabReverse, options);
        }

        /// <summary>
        /// Create a new instance of the <see cref="BertTokenizer"/> class asynchronously.
        /// </summary>
        /// <param name="vocabFilePath">The path to the vocabulary file.</param>
        /// <param name="options">The options to use for the Bert tokenizer.</param>
        /// <param name="cancellationToken">The cancellation token.</param>
        /// <returns>A task that represents the asynchronous creation of the BertTokenizer.</returns>
        /// <remarks>
        /// When creating the tokenizer, ensure that the vocabulary file is sourced from a trusted provider.
        /// </remarks>
        public static async Task<BertTokenizer> CreateAsync(

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Pass a valid opened Stream containing the vocabulary.
2. Check the stream-producing call: File.OpenRead, Assembly.GetManifestResourceStream, etc., for null before calling CreateAsync.
3. Use the vocabFilePath overload CreateAsync(string) which handles opening the file for you.

Example fix

// before
await using var stream = GetVocabStream(); // may return null
var tokenizer = await BertTokenizer.CreateAsync(stream);
// after
await using var stream = GetVocabStream() ?? throw new InvalidOperationException("Vocab stream missing");
var tokenizer = await BertTokenizer.CreateAsync(stream);

if (vocabStream is null) throw new InvalidOperationException("Vocabulary stream must be opened before CreateAsync");

static bool HasVocabStream(Stream? s) => s is not null;

try { var t = await BertTokenizer.CreateAsync(vocabStream, options, ct); } catch (ArgumentNullException ex) when (ex.ParamName == "vocabStream") { /* open the stream and retry once */ }

When it happens

Trigger: Calling CreateAsync((Stream)null) — e.g. passing the result of a stream factory that returned null, or forgetting to open the stream before calling.

Common situations: Reading vocab from embedded resources where GetManifestResourceStream returned null (wrong resource name); streams opened conditionally and not created in some branches; tests passing null streams.

Related errors

• throw new ArgumentNullException(nameof(vocabStream));
• throw new ArgumentNullException(nameof(vocabFilePath));
• throw new ArgumentNullException(nameof(vocabulary));
• throw new ArgumentNullException(nameof(vocabFile));
• throw new ArgumentNullException(nameof(vocabularyPath));

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/6c56630b472c38dd. Report an issue: GitHub.