dotnet/machinelearning · error · ArgumentException

Only one of input_ids or inputs_embeds may be set

Error message

Only one of input_ids or inputs_embeds may be set

What it means

MistralModel.forward, like LlamaModel, treats input_ids and inputs_embeds as mutually exclusive and throws ArgumentException if both are set. Providing both is ambiguous about which representation defines the sequence.

Source

Thrown at src/Microsoft.ML.GenAI.Mistral/MistralModel.cs:70

#pragma warning restore MSML_GeneralName // This name should be PascalCased
    {
        if (input.OverrideCache is not null)
        {
            this._cache = input.OverrideCache;
        }

        var outputAttentions = input.OutputAttentions;
        var outputHiddenStates = input.OutputHiddenStates;
        var attentionMask = input.AttentionMask;
        Device device;
        var inputIds = input.InputIds;
        var positionIds = input.PositionIds;
        var inputsEmbeds = input.InputEmbeddings;
        int batchSize;
        int seqLength;
        if (inputIds is not null && inputsEmbeds is not null)
        {
            throw new ArgumentException("Only one of input_ids or inputs_embeds may be set");
        }
        else if (inputIds is not null)
        {
            batchSize = inputIds.IntShape()[0];
            seqLength = inputIds.IntShape()[1];
            inputsEmbeds = this.embed_tokens.forward(inputIds);
            device = inputIds.device;
        }
        else if (inputsEmbeds is not null)
        {
            batchSize = inputsEmbeds.IntShape()[0];
            seqLength = inputsEmbeds.IntShape()[1];
            device = inputsEmbeds.device;
        }
        else
        {
            throw new ArgumentException("Either input_ids or inputs_embeds must be set");
        }

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Set only one of InputIds or InputEmbeddings before calling forward.
2. Clear InputIds when supplying embeddings (set to null).
3. Make the input construction code choose one representation explicitly.

Example fix

// before
var input = new MistralModelInput { InputIds = ids, InputEmbeddings = embeds };
// after
var input = new MistralModelInput { InputEmbeddings = embeds };

if (input.InputIds is not null && input.InputEmbeddings is not null) throw new ArgumentException("Set only one of InputIds or InputEmbeddings.");

bool HasAmbiguousInput(MistralModelInput i) => i.InputIds is not null && i.InputEmbeddings is not null;

try { output = model.forward(input); } catch (ArgumentException) { /* clear one field and retry */ }

When it happens

Trigger: Calling MistralModel.forward with an input where both InputIds and InputEmbeddings are non-null.

Common situations: Agent/pipeline code that sets embeddings but keeps token ids populated; custom prompt-building code that fills both fields defensively.

Related errors

• Only one of input_ids or inputs_embeds may be set
• Either input_ids or inputs_embeds must be set
• Parameter must not be null, empty, or whitespace
• String.Format(Strings.MismatchedColumnValueType, this.DataTy
• Bad comparer

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/ce8998ce8509acb2. Report an issue: GitHub.