dotnet/machinelearning · error · ArgumentException

Either input_ids or inputs_embeds must be set

Error message

Either input_ids or inputs_embeds must be set

What it means

Phi3Model.forward requires exactly one of input_ids or inputs_embeds. If both are null (neither token ids nor pre-computed embeddings were supplied), neither branch of the if/else executes and the model throws ArgumentException. This is a fail-fast guard because the transformer has nothing to run forward on.

Source

Thrown at src/Microsoft.ML.GenAI.Phi/Module/Phi3Model.cs:90

        {
            throw new ArgumentException("Only one of input_ids or inputs_embeds may be set");
        }
        else if (inputIds is not null)
        {
            batchSize = inputIds.IntShape()[0];
            seqLength = inputIds.IntShape()[1];
            inputsEmbeds = this.embed_tokens.forward(inputIds);
            device = inputIds.device;
        }
        else if (inputsEmbeds is not null)
        {
            batchSize = inputsEmbeds.IntShape()[0];
            seqLength = inputsEmbeds.IntShape()[1];
            device = inputsEmbeds.device;
        }
        else
        {
            throw new ArgumentException("Either input_ids or inputs_embeds must be set");
        }

        var pastKeyValuesLength = input.PastKeyValuesLength;

        if (positionIds is null)
        {
            positionIds = torch.arange(pastKeyValuesLength, seqLength + pastKeyValuesLength, device: device);
            positionIds = positionIds.unsqueeze(0).view(-1, seqLength);
        }
        else
        {
            positionIds = ((long)positionIds.view(-1, seqLength));
        }

        if (this._config.AttnImplementation == "flash_attention_2")
        {
            throw new NotImplementedException();
        }

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Set input_ids on the model input from your tokenizer's encode result before calling forward.
2. If using embeddings, compute inputs_embeds (e.g. via the embedding layer) and assign it to the input.
3. Ensure you are not passing a null tensor due to a failed tokenizer call — check tokenizer output first.

Example fix

// before
var input = new Phi3ModelInput { PastKeyValues = past };
var output = model.forward(input);
// after
var input = new Phi3ModelInput { InputIds = tokenizer.Encode(prompt).ToTensor(), PastKeyValues = past };
var output = model.forward(input);

if (input.InputIds is null && input.InputsEmbeds is null)
    throw new ArgumentException("Provide either input_ids or inputs_embeds before calling forward.");

bool HasModelInput(Phi3ModelInput i) => i?.InputIds is not null || i?.InputsEmbeds is not null;

try { var output = model.forward(input); } catch (ArgumentException ex) when (ex.Message.Contains("input_ids or inputs_embeds")) { /* re-create input with tokenized ids */ }

When it happens

Trigger: Calling the model's forward (directly or via the pipeline/generation loop) with a batch where input_ids is null and inputs_embeds is also null — e.g. constructing the model input manually and forgetting to set input_ids, or a tokenizer step that produced no ids.

Common situations: Hand-building TorchTensor inputs instead of using the tokenizer; refactoring code that previously set inputs_embeds (e.g. for prefix caching) but no longer computes embeddings; calling forward from custom generation code that passes an empty/default input object.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Dimension must be divisible by 2
• Directory "{0}" does not exist.
• Unsupported pixel format
• Invalid width value.
• Invalid height value.

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/50776350d4f524c9. Report an issue: GitHub.