{"record":{"id":"3e1bf0d2f8d3683c","repo":"BerriAI/litellm","slug":"togetherai-does-not-support-multiple-prompts","errorCode":null,"errorMessage":"TogetherAI does not support multiple prompts.","messagePattern":"TogetherAI does not support multiple prompts\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/together_ai/completion/transformation.py","lineNumber":37,"sourceCode":"from ...openai.completion.utils import _transform_prompt\n\n\nclass TogetherAITextCompletionConfig(OpenAITextCompletionConfig):\n    def _transform_prompt(\n        self,\n        messages: list[AllMessageValues] | list[OpenAITextCompletionUserMessage],\n    ) -> AllPromptValues:\n        \"\"\"\n        TogetherAI expects a string prompt.\n        \"\"\"\n        initial_prompt: Final[AllPromptValues] = _transform_prompt(messages)\n        ## TOGETHER AI SPECIFIC VALIDATION ##\n        if isinstance(initial_prompt, list) and is_tokens_or_list_of_tokens(value=initial_prompt):\n            raise ValueError(\"TogetherAI does not support integers as input\")\n        if isinstance(initial_prompt, list) and len(initial_prompt) == 1 and isinstance(initial_prompt[0], str):\n            together_prompt = initial_prompt[0]\n        elif isinstance(initial_prompt, list):\n            raise ValueError(\"TogetherAI does not support multiple prompts.\")\n        else:\n            together_prompt = cast(str, initial_prompt)\n\n        return together_prompt\n\n    def transform_text_completion_request(\n        self,\n        model: str,\n        messages: list[AllMessageValues] | list[OpenAITextCompletionUserMessage],\n        optional_params: dict,\n        headers: dict,\n    ) -> dict:\n        prompt: Final = self._transform_prompt(messages)\n        return {\n            \"model\": model,\n            \"prompt\": prompt,\n            **optional_params,\n        }","sourceCodeStart":19,"sourceCodeEnd":55,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/together_ai/completion/transformation.py#L19-L55","documentation":"TogetherAI's text-completion API takes exactly one string prompt. After LiteLLM converts messages, if the result is a list with more than one element (multiple prompt strings, e.g. batching several prompts in one call), this validation raises. Lists of length one containing a string are unwrapped; anything else list-shaped (other than the token case handled above) fails here.","triggerScenarios":"Calling litellm.text_completion(model=\"together_ai/...\", prompt=[\"prompt A\", \"prompt B\"]) to batch two completions; or messages that transform into a multi-element prompt list (e.g. multiple user contents interpreted as separate prompts).","commonSituations":"Porting batched OpenAI completion code (where prompt can be a list) to TogetherAI; helper libraries that accept List[str] prompts for throughput; n>1 style fan-out mistakenly encoded as multiple prompt strings.","solutions":["Send one prompt string per request; loop or gather over your prompt list instead of passing it in a single call.","Use async concurrency (asyncio.gather over litellm.atext_completion) to keep batch throughput without the list format.","If you passed a single-element list like [\"my prompt\"], unwrap it to the bare string \"my prompt\" (LiteLLM does handle len==1, but explicit strings are clearer)."],"exampleFix":"# before\nresp = litellm.text_completion(\n    model=\"together_ai/togethercomputer/LLaMA-2-7B-32K\",\n    prompt=[\"summarize A\", \"summarize B\"],\n)\n\n# after\nimport asyncio, litellm\nasync def run():\n    return await asyncio.gather(*[\n        litellm.atext_completion(\n            model=\"together_ai/togethercomputer/LLaMA-2-7B-32K\",\n            prompt=p,\n        )\n        for p in [\"summarize A\", \"summarize B\"]\n    ])","handlingStrategy":"type-guard","validationCode":"from typing import Any\n\n\ndef to_single_prompt(prompt: Any) -> str:\n    \"\"\"Normalize prompt for TogetherAI: exactly one string.\"\"\"\n    if isinstance(prompt, list):\n        if len(prompt) == 1 and isinstance(prompt[0], str):\n            return prompt[0]\n        raise ValueError(\"split list prompts into separate together_ai calls\")\n    if not isinstance(prompt, str):\n        raise ValueError(\"together_ai requires a string prompt\")\n    return prompt","typeGuard":"def is_together_safe_prompt(prompt) -> bool:\n    \"\"\"True when prompt is a single string (or 1-element list of str).\"\"\"\n    if isinstance(prompt, str):\n        return True\n    return (\n        isinstance(prompt, list)\n        and len(prompt) == 1\n        and isinstance(prompt[0], str)\n    )","tryCatchPattern":"try:\n    resp = litellm.text_completion(model=\"together_ai/...\", prompt=prompts)\nexcept ValueError as e:\n    if \"does not support multiple prompts\" in str(e) and isinstance(prompts, list):\n        results = [litellm.text_completion(model=\"together_ai/...\", prompt=p) for p in prompts]\n    else:\n        raise","preventionTips":["Never assume OpenAI's list-prompt batching is portable; model your API on str per request.","Use asyncio.gather over atext_completion for batch throughput instead of list prompts.","Add contract tests per provider when a wrapper accepts union input types."],"tags":["together-ai","text-completion","prompt-format","batching","litellm"],"backgroundTag":"invalid-prompt-format","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}