BerriAI/litellm · error · ValueError

TogetherAI does not support multiple prompts.

Error message

TogetherAI does not support multiple prompts.

What it means

TogetherAI's text-completion API takes exactly one string prompt. After LiteLLM converts messages, if the result is a list with more than one element (multiple prompt strings, e.g. batching several prompts in one call), this validation raises. Lists of length one containing a string are unwrapped; anything else list-shaped (other than the token case handled above) fails here.

Source

Thrown at litellm/llms/together_ai/completion/transformation.py:37

from ...openai.completion.utils import _transform_prompt


class TogetherAITextCompletionConfig(OpenAITextCompletionConfig):
    def _transform_prompt(
        self,
        messages: list[AllMessageValues] | list[OpenAITextCompletionUserMessage],
    ) -> AllPromptValues:
        """
        TogetherAI expects a string prompt.
        """
        initial_prompt: Final[AllPromptValues] = _transform_prompt(messages)
        ## TOGETHER AI SPECIFIC VALIDATION ##
        if isinstance(initial_prompt, list) and is_tokens_or_list_of_tokens(value=initial_prompt):
            raise ValueError("TogetherAI does not support integers as input")
        if isinstance(initial_prompt, list) and len(initial_prompt) == 1 and isinstance(initial_prompt[0], str):
            together_prompt = initial_prompt[0]
        elif isinstance(initial_prompt, list):
            raise ValueError("TogetherAI does not support multiple prompts.")
        else:
            together_prompt = cast(str, initial_prompt)

        return together_prompt

    def transform_text_completion_request(
        self,
        model: str,
        messages: list[AllMessageValues] | list[OpenAITextCompletionUserMessage],
        optional_params: dict,
        headers: dict,
    ) -> dict:
        prompt: Final = self._transform_prompt(messages)
        return {
            "model": model,
            "prompt": prompt,
            **optional_params,
        }

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Send one prompt string per request; loop or gather over your prompt list instead of passing it in a single call.
2. Use async concurrency (asyncio.gather over litellm.atext_completion) to keep batch throughput without the list format.
3. If you passed a single-element list like ["my prompt"], unwrap it to the bare string "my prompt" (LiteLLM does handle len==1, but explicit strings are clearer).

Example fix

# before
resp = litellm.text_completion(
    model="together_ai/togethercomputer/LLaMA-2-7B-32K",
    prompt=["summarize A", "summarize B"],
)

# after
import asyncio, litellm
async def run():
    return await asyncio.gather(*[
        litellm.atext_completion(
            model="together_ai/togethercomputer/LLaMA-2-7B-32K",
            prompt=p,
        )
        for p in ["summarize A", "summarize B"]
    ])

from typing import Any


def to_single_prompt(prompt: Any) -> str:
    """Normalize prompt for TogetherAI: exactly one string."""
    if isinstance(prompt, list):
        if len(prompt) == 1 and isinstance(prompt[0], str):
            return prompt[0]
        raise ValueError("split list prompts into separate together_ai calls")
    if not isinstance(prompt, str):
        raise ValueError("together_ai requires a string prompt")
    return prompt

def is_together_safe_prompt(prompt) -> bool:
    """True when prompt is a single string (or 1-element list of str)."""
    if isinstance(prompt, str):
        return True
    return (
        isinstance(prompt, list)
        and len(prompt) == 1
        and isinstance(prompt[0], str)
    )

try:
    resp = litellm.text_completion(model="together_ai/...", prompt=prompts)
except ValueError as e:
    if "does not support multiple prompts" in str(e) and isinstance(prompts, list):
        results = [litellm.text_completion(model="together_ai/...", prompt=p) for p in prompts]
    else:
        raise

When it happens

Trigger: Calling litellm.text_completion(model="together_ai/...", prompt=["prompt A", "prompt B"]) to batch two completions; or messages that transform into a multi-element prompt list (e.g. multiple user contents interpreted as separate prompts).

Common situations: Porting batched OpenAI completion code (where prompt can be a list) to TogetherAI; helper libraries that accept List[str] prompts for throughput; n>1 style fan-out mistakenly encoded as multiple prompt strings.

Related errors

• TogetherAI does not support integers as input
• TogetherAIException - {error_response['error']}
• TogetherAIException - {error_str}
• Batch record for /v1/completions is missing required `prompt
• TogetherAI does not support max_chunks_per_doc

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/3e1bf0d2f8d3683c. Report an issue: GitHub.