dotnet/machinelearning · error · ArgumentException

Either input_ids or inputs_embeds must be set

Error message

Either input_ids or inputs_embeds must be set

What it means

Validation in MistralModel.forward: exactly one of input_ids or inputs_embeds must be supplied. Both set -> "Only one of ..." ArgumentException; neither set -> this error. Without one of these tensors the model cannot determine batch size, sequence length, or device, so the forward pass is rejected.

Source

Thrown at src/Microsoft.ML.GenAI.Mistral/MistralModel.cs:87

        {
            throw new ArgumentException("Only one of input_ids or inputs_embeds may be set");
        }
        else if (inputIds is not null)
        {
            batchSize = inputIds.IntShape()[0];
            seqLength = inputIds.IntShape()[1];
            inputsEmbeds = this.embed_tokens.forward(inputIds);
            device = inputIds.device;
        }
        else if (inputsEmbeds is not null)
        {
            batchSize = inputsEmbeds.IntShape()[0];
            seqLength = inputsEmbeds.IntShape()[1];
            device = inputsEmbeds.device;
        }
        else
        {
            throw new ArgumentException("Either input_ids or inputs_embeds must be set");
        }

        var pastKeyValuesLength = input.PastKeyValuesLength;

        if (positionIds is null)
        {
            positionIds = torch.arange(pastKeyValuesLength, seqLength + pastKeyValuesLength, device: device);
            positionIds = positionIds.unsqueeze(0).view(-1, seqLength);
        }
        else
        {
            positionIds = ((long)positionIds.view(-1, seqLength));
        }

        if (this._config.AttnImplementation == "flash_attention_2")
        {
            throw new NotImplementedException();
        }

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Set InputIds from tokenizer output before calling forward.
2. Or set InputEmbeddings with precomputed embedding tensors.
3. Add an up-front null check on both fields and throw a descriptive error early.

Example fix

// before
var input = new MistralModelInput();
var output = model.forward(input); // throws
// after
var input = new MistralModelInput { InputIds = tokenizer.EncodeToIds(prompt) };
var output = model.forward(input);

if (input.InputIds is null && input.InputEmbeddings is null) throw new ArgumentException("Input requires InputIds or InputEmbeddings.");

bool HasNoInput(MistralModelInput i) => i.InputIds is null && i.InputEmbeddings is null;

try { output = model.forward(input); } catch (ArgumentException) { /* set InputIds from tokenizer output and retry */ }

When it happens

Trigger: Calling MistralModel.forward with both InputIds and InputEmbeddings null — e.g. only PositionIds or AttentionMask was populated on the input.

Common situations: Custom generation loops that forgot to tokenize; deserialized request objects missing the token field; refactor that renamed the ids property without updating the assignment.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Either input_ids or inputs_embeds must be set
• Only one of input_ids or inputs_embeds may be set
• Parameter must not be null, empty, or whitespace
• String.Format(Strings.MismatchedColumnValueType, this.DataTy
• Bad comparer

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/4494e932f103cb37. Report an issue: GitHub.