dotnet/machinelearning · error · ArgumentException

The model type is not Bpe.

Error message

The model type is not Bpe.

What it means

LlamaTokenizer.Create requires a SentencePiece ModelProto whose TrainerSpec.ModelType is Bpe. Passing a model proto trained with a different SentencePiece algorithm (Unigram, Word, Char) throws this ArgumentException.

Source

Thrown at src/Microsoft.ML.Tokenizers/Model/LlamaTokenizer.cs:49

        /// <remarks>
        /// When creating the tokenizer, ensure that the vocabulary stream is sourced from a trusted provider.
        /// </remarks>
        public static new LlamaTokenizer Create(
            Stream modelStream,
            bool addBeginOfSentence = true,
            bool addEndOfSentence = false,
            IReadOnlyDictionary<string, int>? specialTokens = null)
        {
            ModelProto modelProto = ModelProto.Parser.ParseFrom(modelStream);

            if (modelProto is null)
            {
                throw new ArgumentNullException(nameof(modelProto));
            }

            if (modelProto.TrainerSpec.ModelType != TrainerSpec.Types.ModelType.Bpe)
            {
                throw new ArgumentException("The model type is not Bpe.", nameof(modelProto));
            }

            if (modelProto.NormalizerSpec.Name != "identity" && !string.IsNullOrEmpty(modelProto.NormalizerSpec.Name))
            {
                throw new ArgumentException($"Normalization '{modelProto.NormalizerSpec.Name}' is not supported.", nameof(modelProto));
            }

            return new LlamaTokenizer(modelProto, addBeginOfSentence, addEndOfSentence, specialTokens);
        }
    }
}

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Train or download a SentencePiece model with model_type=bpe and pass that proto
2. Inspect proto.TrainerSpec.ModelType before calling Create and route to the matching tokenizer class
3. Regenerate the .model file using sentencepiece with --model_type=bpe

Example fix

// before
var proto = ModelProto.Parser.ParseFrom(File.ReadAllBytes("unigram.model")); // Unigram
var tok = LlamaTokenizer.Create(proto);
// after
// train with: spm_train --model_type=bpe ...
var proto = ModelProto.Parser.ParseFrom(File.ReadAllBytes("bpe.model"));
var tok = LlamaTokenizer.Create(proto);

if (proto.TrainerSpec.ModelType != TrainerSpec.Types.ModelType.Bpe) throw new InvalidOperationException($"Expected Bpe model, got {proto.TrainerSpec.ModelType}");

try { var tok = LlamaTokenizer.Create(proto); } catch (ArgumentException ex) { log.LogError(ex, "SentencePiece model is not Bpe-trained"); throw; }

When it happens

Trigger: Calling LlamaTokenizer.Create(modelProto, ...) with a ModelProto loaded from a .model file that was trained as Unigram (the common SentencePiece default) or Word/Char model.

Common situations: Downloading a generic SentencePiece .model from HuggingFace that is Unigram-based and passing it to LlamaTokenizer; mixing up LLaMA BPE models with T5-style Unigram models.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Normalization '{modelProto.NormalizerSpec.Name}' is not supp
• ArgumentNullException
• The pre_tokenizer 'add_prefix_space' must be a boolean.
• The pre_tokenizer 'replacement' must be a string.
• The Metaspace 'replacement' '{replacement}' is not supported

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/54c1d44a30336137. Report an issue: GitHub.