{"record":{"id":"57fb8771db735c08","repo":"dotnet/machinelearning","slug":"logits-is-null","errorCode":null,"errorMessage":"Logits is null","messagePattern":"Logits is null","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.Core/Pipeline/CausalLMPipeline.cs","lineNumber":160,"sourceCode":"        torch.Tensor? logits = default;\n        var cache = new DynamicKVCache();\n        if (promptLength == totalLen)\n        {\n            var input = new CausalLMModelInput(inputIds, attentionMask, pastKeyValuesLength: 0)\n            {\n                OverrideCache = cache,\n            };\n            var output = this.Model.forward(input);\n            logits = output.Logits;\n        }\n        for (var curPos = promptLength; curPos != totalLen; curPos++)\n        {\n            var input = new CausalLMModelInput(inputIds[.., prevPos..curPos], attentionMask[.., prevPos..curPos], pastKeyValuesLength: prevPos)\n            {\n                OverrideCache = cache,\n            };\n            var output = this.Model.forward(input);\n            logits = output.Logits?.MoveToOtherDisposeScope(inputIds) ?? throw new InvalidOperationException(\"Logits is null\");\n            torch.Tensor nextToken;\n            if (temperature > 0)\n            {\n                var probs = torch.softmax(logits[.., -1] / temperature, dim: -1);\n                nextToken = this.SampleTopP(probs, topP);\n            }\n            else\n            {\n                nextToken = torch.argmax(logits[.., -1], dim: -1);\n            }\n\n            nextToken = nextToken.reshape(-1);\n            inputIds = torch.cat([inputIds, nextToken.unsqueeze(1)], dim: -1).MoveToOtherDisposeScope(inputIds);\n            attentionMask = torch.cat([attentionMask, attentionMask.new_ones(attentionMask.shape[0], 1)], dim: -1);\n            foreach (var stopSequence in stopTokenSequence)\n            {\n                // determine if the last n tokens are the stop sequence\n                var lastN = inputIds[.., ^stopSequence.Length..];","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.Core/Pipeline/CausalLMPipeline.cs#L142-L178","documentation":"During streaming generation, the model's forward output must carry a Logits tensor; if output.Logits is null the pipeline cannot sample the next token and throws InvalidOperationException after moving tensors to the correct dispose scope.","triggerScenarios":"Calling GenerateStreaming (directly or via Generate) when Model.forward returns a CausalLMModelOutput whose Logits property is null for the given input shape/cache configuration.","commonSituations":"Custom model implementation returning output without logits; model/forward wrapper mismatch where the head is disabled; incorrect CausalLMModelInput construction causing the model to skip logits computation.","solutions":["Verify the loaded model's forward always populates Logits in CausalLMModelOutput","Check that the model head (lm_head) is loaded and attached","Use a stock model implementation for the checkpoint rather than a custom forward override"],"exampleFix":"// before\nvar output = model.forward(input); // custom impl returns Logits = null\n// after\nclass MyOutput : CausalLMModelOutput {\n    public override Tensor Logits => lmHead(lastHiddenState); // ensure logits populated\n}","handlingStrategy":"try-catch","validationCode":"var probe = Model.forward(CausalLMModelInput.CreateTest(batch:1, seq:1));\nif (probe.Logits is null) throw new InvalidOperationException(\"Model forward does not produce logits\");","typeGuard":"bool HasLogits(CausalLMModelOutput o) => o?.Logits is not null && !o.Logits.IsInvalid;","tryCatchPattern":"try { await foreach (var t in pipeline.GenerateStreaming(input, mask, stopTokens)) { /* consume */ } } catch (InvalidOperationException ex) when (ex.Message == \"Logits is null\") { // fallback to a known-good model implementation }","preventionTips":["Use stock model implementations matching the checkpoint architecture","Verify lm_head is loaded during weight loading","Test a single-token forward before running long generations"],"tags":["genai","llm","null-result"],"backgroundTag":"unexpected-api-response-shape","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}