dotnet/machinelearning · error · ArgumentException

Either input_ids or inputs_embeds must be set

Error message

Either input_ids or inputs_embeds must be set

What it means

Validation in LlamaModel.forward: exactly one of input_ids or inputs_embeds must be supplied. If both are given, a separate ArgumentException fires ("Only one of ..."); if neither is given, execution reaches the final else and throws this error stating that at least one is required, since the forward pass has no input to embed or attend over.

Source

Thrown at src/Microsoft.ML.GenAI.LLaMA/Module/LlamaModel.cs:93

        {
            throw new ArgumentException("Only one of input_ids or inputs_embeds may be set");
        }
        else if (inputIds is not null)
        {
            batchSize = inputIds.IntShape()[0];
            seqLength = inputIds.IntShape()[1];
            inputsEmbeds = this.embed_tokens.forward(inputIds);
            device = inputIds.device;
        }
        else if (inputsEmbeds is not null)
        {
            batchSize = inputsEmbeds.IntShape()[0];
            seqLength = inputsEmbeds.IntShape()[1];
            device = inputsEmbeds.device;
        }
        else
        {
            throw new ArgumentException("Either input_ids or inputs_embeds must be set");
        }

        var pastKeyValuesLength = input.PastKeyValuesLength;

        if (positionIds is null)
        {
            positionIds = torch.arange(pastKeyValuesLength, seqLength + pastKeyValuesLength, device: device);
            positionIds = positionIds.unsqueeze(0).view(-1, seqLength);
        }
        else
        {
            positionIds = ((long)positionIds.view(-1, seqLength));
        }

        if (this._config.AttnImplementation == "flash_attention_2")
        {
            throw new NotImplementedException();
        }

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Set InputIds (token ids from the tokenizer) on the input object.
2. Alternatively set InputEmbeddings to precomputed embeddings.
3. Add a null check on the input before calling forward to fail fast with a clearer message.

Example fix

// before
var input = new LlamaModelInput { AttentionMask = mask };
var output = model.forward(input); // throws
// after
var input = new LlamaModelInput { InputIds = tokenizer.Encode(prompt), AttentionMask = mask };
var output = model.forward(input);

if (input.InputIds is null && input.InputEmbeddings is null) throw new ArgumentException("Input requires InputIds or InputEmbeddings.");

bool HasNoInput(LlamaModelInput i) => i.InputIds is null && i.InputEmbeddings is null;

try { output = model.forward(input); } catch (ArgumentException) { /* tokenize the prompt and set InputIds, then retry */ }

When it happens

Trigger: Calling LlamaModel.forward with a LlamaModelInput whose InputIds and InputEmbeddings are both null (e.g. only PositionIds/attention mask set).

Common situations: Building custom inference loops where the input struct is created empty and fields are filled conditionally, with a branch that never executes; or deserializing a request that dropped the token field.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Only one of input_ids or inputs_embeds may be set
• Either input_ids or inputs_embeds must be set
• Parameter must not be null, empty, or whitespace
• String.Format(Strings.MismatchedColumnValueType, this.DataTy
• Bad comparer

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/8e60a704ed3fa3fc. Report an issue: GitHub.