microsoft/autogen · error · Exception

agent fails to generate json response

Error message

agent fails to generate json response

What it means

Thrown by a custom middleware in MiddlewareAgentCodeSnippet that enforces JSON-formatted replies: after up to two attempts (original prompt plus a 'reply again in json' review prompt), the reply still wasn't valid JSON, so the middleware gives up and throws. It demonstrates a response-format enforcement pattern where failure to comply is a hard error, not a retry-forever loop.

Source

Thrown at dotnet/samples/AgentChat/AutoGen.Basic.Sample/CodeSnippet/MiddlewareAgentCodeSnippet.cs:173

            {
                if (JsonSerializer.Deserialize<Dictionary<string, object>>(reply.GetContent()) is { } dict)
                {
                    return reply;
                }
                else
                {
                    await Task.Delay(1000);
                    var reviewPrompt = @"The format is not json, please modify your response to json format
        -- ORIGINAL MESSAGE --
        {reply.Content}
        -- END OF ORIGINAL MESSAGE --
        
        Reply again with json format.";
                    reply = await agent.SendAsync(reviewPrompt, messages, ct);
                }
            }

            throw new Exception("agent fails to generate json response");
        });
        #endregion code_snippet_response_format_forcement
    }
}

View on GitHub (pinned to 027ecf0a37)

Solutions

1. Catch the exception at the caller and re-ask with an even more explicit instruction (provide a schema/example in the prompt).
2. Switch to a model that supports native structured/JSON output (e.g. response_format json_object / json_schema) instead of prompt-based enforcement.
3. Strip markdown fences before parsing: reply.Content.replace(/^```(json)?|```$/g, '').
4. Increase max_tokens so the JSON response isn't truncated.

Example fix

// before
reply = await agent.SendAsync(reviewPrompt, messages, ct);
// (loop ends, still not json)
throw new Exception("agent fails to generate json response");

// after (tolerant parse before giving up)
var content = (reply.Content ?? "").Trim()
    .Replace("```json", "").Replace("```", "").Trim();
try { JsonDocument.Parse(content); return reply; }
catch { throw new Exception("agent fails to generate json response"); }

bool IsJson(string? s)
{
    if (string.IsNullOrWhiteSpace(s)) return false;
    try { System.Text.Json.JsonDocument.Parse(s!); return true; }
    catch { return false; }
}

try
{
    reply = await jsonEnforcingAgent.SendAsync(prompt, ct: ct);
}
catch (Exception e) when (e.Message.Contains("agent fails to generate json response"))
{
    // Bounded retries are exhausted: degrade to a non-JSON path or re-prompt the user
    reply = await fallbackPlainAgent.SendAsync(prompt, ct: ct);
}

When it happens

Trigger: Calling the wrapped agent where the LLM repeatedly returns prose/markdown instead of pure JSON — e.g. wrapping JSON in ```json fences, adding commentary, or the model being too weak to follow the format instruction. The retry loop is bounded, so a persistently non-compliant model exhausts it.

Common situations: Using a small model that ignores JSON format instructions; prompt competing with other instructions ('explain your answer'); JSON content containing characters the model escapes incorrectly; max-token truncation cutting the JSON mid-stream.

Related errors

• Please set OPENAI_API_KEY environment variable.
• Please set ANTHROPIC_API_KEY environment variable.
• Missing ANTHROPIC_API_KEY environment variable.
• Missing ANTHROPIC_API_KEY environment variable.
• Please set OPENAI_API_KEY environment variable.

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/6748893663b0303c. Report an issue: GitHub.