dotnet/machinelearning · error · InvalidOperationException

Failed to deserialize tool calls.

Error message

Failed to deserialize tool calls.

What it means

ParseAsToolCallMessage strips the "[TOOL_CALLS]" prefix and JSON-deserializes the remainder into List<MistralToolCall>; JsonSerializer.Deserialize can return null (or the JSON shape mismatches) and the code throws InvalidOperationException. This indicates the model's tool-call output was not in the expected Mistral format.

Source

Thrown at src/Microsoft.ML.GenAI.Mistral/MistralCausalLMAgent.cs:149

        }
    }

    private class MistralToolCall
    {
        [JsonPropertyName("name")]
        public string? Name { get; set; }

        [JsonPropertyName("arguments")]
        public JsonObject? Arguments { get; set; }
    }

    private ToolCallMessage ParseAsToolCallMessage(string content)
    {
        var json = content.Substring("[TOOL_CALLS]".Length).Trim();

        // the json string should be a list of tool call messages
        // e.g. [{"name": "get_current_weather", "parameters": {"location": "Seattle"}}]
        var mistralToolCalls = JsonSerializer.Deserialize<List<MistralToolCall>>(json) ?? throw new InvalidOperationException("Failed to deserialize tool calls.");
        var toolCalls = mistralToolCalls
            .Select(tc => new ToolCall(tc.Name!, JsonSerializer.Serialize(tc.Arguments)) { ToolCallId = this.GenerateToolCallId() });

        return new ToolCallMessage(toolCalls, from: this.Name);
    }

    /// <summary>
    /// 9 random alphanumeric characters
    /// </summary>
    private string GenerateToolCallId(int length = 9)
    {
        const string chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
        var random = new Random();
        return new string(Enumerable.Repeat(chars, length)
          .Select(s => s[random.Next(s.Length)]).ToArray());
    }
}

View on GitHub (pinned to 7b76e69cf9)

Solutions

1. Increase maxLen / adjust stop sequences so the JSON tool call is not truncated.
2. Validate/log the raw output before parsing; if the shape differs (object not array, wrong keys), adapt parsing or update the chat template/prompt to enforce Mistral's tool-call format.
3. Wrap the parse in try-catch on JsonException and InvalidOperation to fall back to a plain assistant message.

Example fix

// before
var msg = await agent.GenerateReplyAsync(history, tools: tools); // throws on malformed JSON
// after
try { var msg = await agent.GenerateReplyAsync(history, tools: tools); }
catch (InvalidOperationException) { /* fall back: re-prompt model to emit valid [TOOL_CALLS] JSON */ }

var json = content.StartsWith("[TOOL_CALLS]") ? content.Substring(12).Trim() : null;
bool IsParseableToolCalls(string? json) => json is not null && json.StartsWith("[") && json.EndsWith("]");

try { return agent.GenerateReplyAsync(history, options: opts); } catch (InvalidOperationException ex) when (ex.Message.Contains("Failed to deserialize tool calls")) { /* treat output as plain text or re-prompt */ }

When it happens

Trigger: Model output starting with "[TOOL_CALLS]" but whose remainder is not a JSON array of {name, arguments} objects — e.g. malformed/truncated JSON, a JSON object instead of a list, or different field names.

Common situations: Model hallucinating a tool-call format from its chat template; output truncated by maxLen or stop sequences mid-JSON; using a non-Mistral finetune with a different tool-call syntax.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• Failed to generate a reply.
• Only one of input_ids or inputs_embeds may be set
• Either input_ids or inputs_embeds must be set
• Please provide at most one system message.
• unknown schema type: {schema.Type}

AI-assisted analysis of dotnet/machinelearning@7b76e69cf9 (2026-09-11). Data as JSON: /api/errors/d900c2d4d43e757c. Report an issue: GitHub.