dotnet/machinelearning · error · ArgumentException

Could not find the vocabulary file

Error message

Could not find the vocabulary file '{vocabFile}'.

What it means

BpeOptions checks File.Exists(vocabFile) and throws ArgumentException when the supplied vocabulary file path does not exist on disk. The library requires the vocab JSON at construction because it immediately opens and deserializes it. This surfaces a bad path as a clear error instead of an obscure IOException from File.OpenRead.

Solutions

1. Verify the file exists at the exact path with File.Exists before constructing BpeOptions.
2. Use an absolute path or AppContext.BaseDirectory-relative path so the lookup does not depend on the current working directory.
3. Ensure the vocab.json asset is deployed/copied with your application output.
4. Confirm the file name and extension match what the model expects (e.g. vocab.json, not vocab.txt).

Example fix

// before
var options = new BpeOptions("vocab.json");
// after
string vocabPath = Path.Combine(AppContext.BaseDirectory, "models", "vocab.json");
if (!File.Exists(vocabPath)) throw new FileNotFoundException("BPE vocab missing", vocabPath);
var options = new BpeOptions(vocabPath);

if (!File.Exists(vocabPath)) throw new FileNotFoundException("BPE vocab file not found", vocabPath);

try { var options = new BpeOptions(vocabPath); } catch (ArgumentException ex) { logger.LogError(ex, "Vocab file missing at {Path}", vocabPath); }

When it happens

Trigger: new BpeOptions(@"models\vocab.json") where the file is absent, the working directory differs from the expected one, or the path has a typo or wrong casing/extension.

Common situations: Model files not copied to the output directory (missing CopyToOutputDirectory in csproj); relative path resolved against a different CWD when running under a test runner or service; downloading model assets was skipped.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• Could not find the merges file
• argument should not be null.
• Cannot read the file Merge file.
• Cannot read the file Merge file.
• Failed to read the vocabulary file.

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/1c46b6ad4a94a33f. Report an issue: GitHub.

Appendix: source

Thrown at src/Microsoft.ML.Tokenizers/Model/BpeOptions.cs:46

            Vocabulary = vocabulary;
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="BpeOptions"/> class.
        /// </summary>
        /// <param name="vocabFile">The JSON file path containing the dictionary of string keys and their ids.</param>
        /// <param name="mergesFile">The file path containing the tokens's pairs list.</param>
        public BpeOptions(string vocabFile, string? mergesFile = null)
        {
            if (vocabFile is null)
            {
                throw new ArgumentNullException(nameof(vocabFile));
            }

            if (!File.Exists(vocabFile))
            {
                throw new ArgumentException($"Could not find the vocabulary file '{vocabFile}'.");
            }

            using Stream vocabStream = File.OpenRead(vocabFile);
            Dictionary<string, int>? dictionary = JsonSerializer.Deserialize<Dictionary<string, int>>(vocabStream);

            if (dictionary is null)
            {
                throw new InvalidOperationException($"The content of the vocabulary file '{vocabFile}' is not valid.");
            }

            Vocabulary = dictionary;

            if (mergesFile is not null)
            {
                if (!File.Exists(mergesFile))
                {
                    throw new ArgumentException($"Could not find the merges file '{mergesFile}'.");
                }

View on GitHub (pinned to 7b76e69cf9)