{"record":{"id":"38f8df20ba478ef2","repo":"hiyouga/LlamaFactory","slug":"cannot-stream-function-calls","errorCode":null,"errorMessage":"Cannot stream function calls.","messagePattern":"Cannot stream function calls\\.","errorType":"http","errorClass":"HTTPException","httpStatus":400,"severity":"error","filePath":"src/llamafactory/api/chat.py","lineNumber":253,"sourceCode":"        prompt_length = response.prompt_length\n        response_length += response.response_length\n\n    usage = ChatCompletionResponseUsage(\n        prompt_tokens=prompt_length,\n        completion_tokens=response_length,\n        total_tokens=prompt_length + response_length,\n    )\n\n    return ChatCompletionResponse(id=completion_id, model=request.model, choices=choices, usage=usage)\n\n\nasync def create_stream_chat_completion_response(\n    request: \"ChatCompletionRequest\", chat_model: \"ChatModel\"\n) -> AsyncGenerator[str, None]:\n    completion_id = f\"chatcmpl-{uuid.uuid4().hex}\"\n    input_messages, system, tools, images, videos, audios = _process_request(request)\n    if tools:\n        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=\"Cannot stream function calls.\")\n\n    if request.n > 1:\n        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=\"Cannot stream multiple responses.\")\n\n    yield _create_stream_chat_completion_chunk(\n        completion_id=completion_id, model=request.model, delta=ChatCompletionMessage(role=Role.ASSISTANT, content=\"\")\n    )\n    async for new_token in chat_model.astream_chat(\n        input_messages,\n        system,\n        tools,\n        images,\n        videos,\n        audios,\n        do_sample=request.do_sample,\n        temperature=request.temperature,\n        top_p=request.top_p,\n        max_new_tokens=request.max_tokens,","sourceCodeStart":235,"sourceCodeEnd":271,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/api/chat.py#L235-L271","documentation":"Raised as HTTP 400 by the streaming endpoint (stream: true) when the request carries a non-empty tools list. LlamaFactory's stream path does not implement tool-call streaming, so function-calling requests must use the non-streaming endpoint. The check happens after _process_request, before any chunk is yielded.","triggerScenarios":"POST /v1/chat/completions with stream: true and a non-empty tools array; client SDKs (e.g. openai-python with stream=True + tools) that always set both.","commonSituations":"Copy-pasting an OpenAI tool-use streaming example against LlamaFactory; toggling stream=true for latency while forgetting tools are set on the client.","solutions":["Set stream: false when sending tools.","Or drop the tools array if function calling is not actually needed for this request.","Wrap the streaming call and fall back to non-streaming when this 400 is returned.","Track LlamaFactory releases for tool-call streaming support before re-enabling."],"exampleFix":"// before\nconst res = await client.chat.completions.create({ model, messages, tools, stream: true });\n// after\nconst res = await client.chat.completions.create({ model, messages, tools, stream: false });","handlingStrategy":"validation","validationCode":"def can_stream(request):\n    return not (request.get(\"tools\") and request.get(\"stream\"))\n\nif not can_stream(payload):\n    payload = {**payload, \"stream\": False}","typeGuard":"const canStream = (req) => !req.tools?.length || !req.stream;","tryCatchPattern":"try { stream(...) } catch (e) { if (e.status === 400 && e.detail === 'Cannot stream function calls.') { return nonStream({...req, stream: false}); } throw e; }","preventionTips":["Gate streaming on !tools in the request builder.","Watch LlamaFactory release notes for tool-call streaming support.","Add an integration test covering the tools+stream combination."],"tags":["api","streaming","tools","http-400","function-calling"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}