What causes "Only one of input_ids or inputs_embeds may be set" in dotnet/machinelearning?

Pass only one: null out inputIds when supplying InputEmbeddings, or vice versa

dotnet/machinelearning · error · ArgumentException

Only one of input_ids or inputs_embeds may be set

Error message

Only one of input_ids or inputs_embeds may be set

What it means

Phi3Model.forward accepts either input token ids or precomputed input embeddings, but not both. Supplying both triggers ArgumentException('Only one of input_ids or inputs_embeds may be set'), mirroring the transformers mutual-exclusion check.

Source

Thrown at src/Microsoft.ML.GenAI.Phi/Module/Phi3Model.cs:73

#pragma warning restore MSML_GeneralName // This name should be PascalCased
    {
        if (input.OverrideCache is not null)
        {
            this._cache = input.OverrideCache;
        }

        var outputAttentions = input.OutputAttentions;
        var outputHiddenStates = input.OutputHiddenStates;
        var attentionMask = input.AttentionMask;
        Device device;
        var inputIds = input.InputIds;
        var positionIds = input.PositionIds;
        var inputsEmbeds = input.InputEmbeddings;
        int batchSize;
        int seqLength;
        if (inputIds is not null && inputsEmbeds is not null)
        {
            throw new ArgumentException("Only one of input_ids or inputs_embeds may be set");
        }
        else if (inputIds is not null)
        {
            batchSize = inputIds.IntShape()[0];
            seqLength = inputIds.IntShape()[1];
            inputsEmbeds = this.embed_tokens.forward(inputIds);
            device = inputIds.device;
        }
        else if (inputsEmbeds is not null)
        {
            batchSize = inputsEmbeds.IntShape()[0];
            seqLength = inputsEmbeds.IntShape()[1];
            device = inputsEmbeds.device;
        }
        else
        {
            throw new ArgumentException("Either input_ids or inputs_embeds must be set");
        }

View on GitHub (pinned to 7b76e69cf9)

Solutions

Pass only one: null out inputIds when supplying InputEmbeddings, or vice versa
If you want embedding-based input, clear InputIds on the input struct/record
Add a pre-call check mirroring the mutual exclusion and choose ids as the canonical input

Example fix

// before
model.forward(inputIds: ids, input: new Phi3Input { InputEmbeddings = embeds, ... });
// after
model.forward(inputIds: null, input: new Phi3Input { InputEmbeddings = embeds, ... });

Defensive patterns

Strategy: validation

Validate before calling

if (inputIds is not null && input.InputEmbeddings is not null)
    inputIds = null; // or throw, depending on desired strictness

Type guard

static bool HasExactlyOneInput(TorchTensor? ids, TorchTensor? embeds) => (ids is null) ^ (embeds is null);

Try / catch

try { output = model.forward(inputIds, input, options); }
catch (ArgumentException ex) when (ex.Message.Contains("Only one of input_ids or inputs_embeds")) { throw new InvalidOperationException("Provide either input_ids or inputs_embeds, not both", ex); }

Prevention

Choose one canonical input path (ids) unless you explicitly need embeddings
Clear InputIds when supplying InputEmbeddings in pipeline code
Assert mutual exclusion at the boundary before calling the model

When it happens

Trigger: Calling Phi3Model.forward with input.InputEmbeddings set and inputIds also set (non-null) — e.g. a pipeline that precomputes embeddings while still passing token ids.

Common situations: Caching embeddings for speed while also forwarding ids; generic inference harnesses that populate both fields by default; migration from Phi2Model (which rejected embeddings entirely) while leaving ids in place.

Related errors

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/4537b16cf27fa289. Report an issue: GitHub.