dotnet/machinelearning · error · NotImplementedException

inputEmbeddings is not supported

Error message

inputEmbeddings is not supported

What it means

Phi2Model.forward does not support passing precomputed input embeddings; if inputEmbeddings is non-null it throws NotImplementedException('inputEmbeddings is not supported'). Embeddings are always computed internally from inputIds.

Source

Thrown at src/Microsoft.ML.GenAI.Phi/Module/Phi2Model.cs:66

    public Phi2Config Config => this._config;

#pragma warning disable MSML_GeneralName // This name should be PascalCased
    public override (Tensor, Tensor?, Tensor?) forward(
#pragma warning restore MSML_GeneralName // This name should be PascalCased
        Tensor inputIds,
        Tensor? attentionMask = null,
        int pastKeyValueLength = 0,
        Tensor? positionIds = null,
        Tensor? inputEmbeddings = null,
        (bool, bool, bool) options = default) // use_cache, output_attentions, output_hidden_states
    {
        (var outputAttentions, var outputHiddenStates, var useCache) = options;

        // TODO
        // add support for inputEmbeddings
        if (inputEmbeddings is not null)
        {
            throw new NotImplementedException("inputEmbeddings is not supported");
        }
        inputEmbeddings = this.embed_tokens.forward(inputIds);
        inputEmbeddings = this.embed_dropout.forward(inputEmbeddings);
        var batchSize = inputIds.shape[0];
        var seqLen = (int)inputIds.shape[1];

        if (positionIds is null)
        {
            positionIds = torch.arange(pastKeyValueLength, seqLen + pastKeyValueLength, dtype: inputIds.dtype, device: inputIds.device);
            positionIds = positionIds.unsqueeze(0);
        }

        // attention
        // use 4d attention mask
        if (attentionMask is not null)
        {
            attentionMask = this.Prepare4DCausalAttentionMask(attentionMask, seqLen, pastKeyValueLength, inputEmbeddings.dtype);
        }

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Pass inputIds and let the model compute embeddings itself
2. Tokenize your inputs before calling forward instead of pre-embedding them
3. Implement embedding-input support in Phi2Model (skip embed_tokens when embeddings supplied) if you own the code

Example fix

// before
model.forward(inputIds: null, inputEmbeddings: embeddings, ...);
// after
var ids = tokenizer.Encode(text); model.forward(inputIds: ids, inputEmbeddings: null, ...);

if (embeddings is not null) throw new InvalidOperationException("Phi2Model does not accept input embeddings; pass inputIds instead");

try { output = model.forward(inputIds: ids, inputEmbeddings: null, options); }
catch (NotImplementedException ex) when (ex.Message.Contains("inputEmbeddings")) { throw new NotSupportedException("Tokenize inputs before calling Phi2Model.forward", ex); }

When it happens

Trigger: Calling Phi2Model.forward with both/independent inputEmbeddings, e.g. continuing generation from hidden states, passing decoder inputs_embeds, or a generic pipeline that always supplies embeddings.

Common situations: Porting HuggingFace code that uses inputs_embeds; piping outputs of one model into Phi-2 without tokenizing; shared inference harness passing null inputIds with non-null embeddings.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• {nameof(ChannelMessageKind)}.{e.Kind} is not yet implemented
• Metric {typeof(TMetrics)} not implemented
• num_key_value_heads must be specified
• Not implemented type {typeof(T)}
• {fieldType.Name}

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/0dc855ef8288dd86. Report an issue: GitHub.