{"record":{"id":"50776350d4f524c9","repo":"dotnet/machinelearning","slug":"either-input-ids-or-inputs-embeds-must-be-set-507763","errorCode":null,"errorMessage":"Either input_ids or inputs_embeds must be set","messagePattern":"Either input_ids or inputs_embeds must be set","errorType":"exception","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.Phi/Module/Phi3Model.cs","lineNumber":90,"sourceCode":"        {\n            throw new ArgumentException(\"Only one of input_ids or inputs_embeds may be set\");\n        }\n        else if (inputIds is not null)\n        {\n            batchSize = inputIds.IntShape()[0];\n            seqLength = inputIds.IntShape()[1];\n            inputsEmbeds = this.embed_tokens.forward(inputIds);\n            device = inputIds.device;\n        }\n        else if (inputsEmbeds is not null)\n        {\n            batchSize = inputsEmbeds.IntShape()[0];\n            seqLength = inputsEmbeds.IntShape()[1];\n            device = inputsEmbeds.device;\n        }\n        else\n        {\n            throw new ArgumentException(\"Either input_ids or inputs_embeds must be set\");\n        }\n\n        var pastKeyValuesLength = input.PastKeyValuesLength;\n\n        if (positionIds is null)\n        {\n            positionIds = torch.arange(pastKeyValuesLength, seqLength + pastKeyValuesLength, device: device);\n            positionIds = positionIds.unsqueeze(0).view(-1, seqLength);\n        }\n        else\n        {\n            positionIds = ((long)positionIds.view(-1, seqLength));\n        }\n\n        if (this._config.AttnImplementation == \"flash_attention_2\")\n        {\n            throw new NotImplementedException();\n        }","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.Phi/Module/Phi3Model.cs#L72-L108","documentation":"Phi3Model.forward requires exactly one of input_ids or inputs_embeds. If both are null (neither token ids nor pre-computed embeddings were supplied), neither branch of the if/else executes and the model throws ArgumentException. This is a fail-fast guard because the transformer has nothing to run forward on.","triggerScenarios":"Calling the model's forward (directly or via the pipeline/generation loop) with a batch where input_ids is null and inputs_embeds is also null — e.g. constructing the model input manually and forgetting to set input_ids, or a tokenizer step that produced no ids.","commonSituations":"Hand-building TorchTensor inputs instead of using the tokenizer; refactoring code that previously set inputs_embeds (e.g. for prefix caching) but no longer computes embeddings; calling forward from custom generation code that passes an empty/default input object.","solutions":["Set input_ids on the model input from your tokenizer's encode result before calling forward.","If using embeddings, compute inputs_embeds (e.g. via the embedding layer) and assign it to the input.","Ensure you are not passing a null tensor due to a failed tokenizer call — check tokenizer output first."],"exampleFix":"// before\nvar input = new Phi3ModelInput { PastKeyValues = past };\nvar output = model.forward(input);\n// after\nvar input = new Phi3ModelInput { InputIds = tokenizer.Encode(prompt).ToTensor(), PastKeyValues = past };\nvar output = model.forward(input);","handlingStrategy":"validation","validationCode":"if (input.InputIds is null && input.InputsEmbeds is null)\n    throw new ArgumentException(\"Provide either input_ids or inputs_embeds before calling forward.\");","typeGuard":"bool HasModelInput(Phi3ModelInput i) => i?.InputIds is not null || i?.InputsEmbeds is not null;","tryCatchPattern":"try { var output = model.forward(input); } catch (ArgumentException ex) when (ex.Message.Contains(\"input_ids or inputs_embeds\")) { /* re-create input with tokenized ids */ }","preventionTips":["Always build model inputs via the tokenizer/processor helpers rather than by hand.","Assert input.InputIds is not null in unit tests for generation code paths.","Never reuse an input object after clearing its tensors."],"tags":["csharp","machine-learning","argument-validation","tensor"],"backgroundTag":"missing-required-argument","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}