{"record":{"id":"21db878efc2e0876","repo":"dotnet/machinelearning","slug":"failed-to-generate-a-reply-21db87","errorCode":null,"errorMessage":"Failed to generate a reply.","messagePattern":"Failed to generate a reply\\.","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"error","filePath":"src/Microsoft.ML.GenAI.Phi/Phi3/Phi3CausalLMAgent.cs","lineNumber":56,"sourceCode":"    public Task<IMessage> GenerateReplyAsync(IEnumerable<IMessage> messages, GenerateReplyOptions? options = null, CancellationToken cancellationToken = default)\n    {\n        if (_systemMessage != null)\n        {\n            var systemMessage = new TextMessage(Role.System, _systemMessage, from: this.Name);\n            messages = messages.Prepend(systemMessage);\n        }\n\n        var input = _templateBuilder.BuildPrompt(messages);\n        var maxLen = options?.MaxToken ?? 1024;\n        var temperature = options?.Temperature ?? 0.7f;\n        var stopTokenSequence = options?.StopSequence ?? [];\n        stopTokenSequence = stopTokenSequence.Append(\"<|end|>\").ToArray();\n\n        var output = _pipeline.Generate(\n            input,\n            maxLen: maxLen,\n            temperature: temperature,\n            stopSequences: stopTokenSequence) ?? throw new InvalidOperationException(\"Failed to generate a reply.\");\n\n        return Task.FromResult<IMessage>(new TextMessage(Role.Assistant, output, from: this.Name));\n    }\n\n#pragma warning disable CS1998 // Async method lacks 'await' operators and will run synchronously\n    public async IAsyncEnumerable<IMessage> GenerateStreamingReplyAsync(\n#pragma warning restore CS1998 // Async method lacks 'await' operators and will run synchronously\n        IEnumerable<IMessage> messages,\n        GenerateReplyOptions? options = null,\n        [EnumeratorCancellation] CancellationToken cancellationToken = default)\n    {\n        if (_systemMessage != null)\n        {\n            var systemMessage = new TextMessage(Role.System, _systemMessage, from: this.Name);\n            messages = messages.Prepend(systemMessage);\n        }\n\n        var input = _templateBuilder.BuildPrompt(messages);","sourceCodeStart":38,"sourceCodeEnd":74,"githubUrl":"https://github.com/dotnet/machinelearning/blob/7b76e69cf964daeca3f1377af6bc5543284d56c6/src/Microsoft.ML.GenAI.Phi/Phi3/Phi3CausalLMAgent.cs#L38-L74","documentation":"Phi3CausalLMAgent.GenerateReplyAsync calls the underlying pipeline's Generate, which returns null when no text could be produced. The agent converts that null into InvalidOperationException('Failed to generate a reply.') so callers get a clear failure instead of a null message.","triggerScenarios":"Calling GenerateReplyAsync with an empty prompt, a stop sequence that matches the very first generated token (note '<|end|>' is always appended), or a maxLen/temperature configuration that yields no output tokens.","commonSituations":"Prompt built from an empty chat history; stopSequences supplied by the caller already containing '<|end|>' plus overly aggressive custom stops; model weights/tokenizer mismatch producing garbage that the pipeline filters out; maxLen set too small to produce any token.","solutions":["Inspect the input prompt/chat template — ensure it is non-empty and correctly formatted for Phi-3.","Remove or relax custom stopSequences so generation is not stopped before any token is emitted.","Increase maxLen (and verify temperature is in a valid range) and retry.","Verify the model weights and tokenizer files load correctly and match the Phi-3 version you target."],"exampleFix":"// before\nvar reply = await agent.GenerateReplyAsync(history, new Phi3AgentRequestOptions { MaxLen = 1 });\n// after\nvar reply = await agent.GenerateReplyAsync(history, new Phi3AgentRequestOptions { MaxLen = 1024 });","handlingStrategy":"try-catch","validationCode":"if (string.IsNullOrWhiteSpace(prompt) && history.All(m => string.IsNullOrWhiteSpace(m.GetContent())))\n    throw new InvalidOperationException(\"Chat history is empty; generation would produce no output.\");","typeGuard":null,"tryCatchPattern":"try { var reply = await agent.GenerateReplyAsync(history, options); } catch (InvalidOperationException ex) when (ex.Message == \"Failed to generate a reply.\") { logger.LogWarning(ex, \"Generation produced no output\"); return fallbackMessage; }","preventionTips":["Keep maxLen comfortably above the expected reply length.","Avoid stop sequences that could match the first generated token.","Log the fully rendered prompt when generation fails so it can be inspected.","Verify model and tokenizer files load before serving requests."],"tags":["csharp","llm","generation","null-result"],"backgroundTag":"empty-result-set","analyzedSha":"7b76e69cf964daeca3f1377af6bc5543284d56c6","analyzedAt":"2026-09-11T12:35:38.930Z","contentChangedAt":"2026-09-11T12:35:38.930Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}