BerriAI/litellm · error · ValueError

TogetherAI does not support integers as input

Error message

TogetherAI does not support integers as input

What it means

TogetherAI's text-completion endpoint accepts a plain string prompt only. LiteLLM first converts messages via _transform_prompt, and if the result is a list of token IDs (integers) — e.g. messages supplied with precomputed 'tokens'/'prompt_token_ids' — this validation raises immediately. TogetherAI has no token-list input mode, so the request can never be sent.

Source

Thrown at litellm/llms/together_ai/completion/transformation.py:33

    OpenAITextCompletionUserMessage,
)

from ...openai.completion.transformation import OpenAITextCompletionConfig
from ...openai.completion.utils import _transform_prompt


class TogetherAITextCompletionConfig(OpenAITextCompletionConfig):
    def _transform_prompt(
        self,
        messages: list[AllMessageValues] | list[OpenAITextCompletionUserMessage],
    ) -> AllPromptValues:
        """
        TogetherAI expects a string prompt.
        """
        initial_prompt: Final[AllPromptValues] = _transform_prompt(messages)
        ## TOGETHER AI SPECIFIC VALIDATION ##
        if isinstance(initial_prompt, list) and is_tokens_or_list_of_tokens(value=initial_prompt):
            raise ValueError("TogetherAI does not support integers as input")
        if isinstance(initial_prompt, list) and len(initial_prompt) == 1 and isinstance(initial_prompt[0], str):
            together_prompt = initial_prompt[0]
        elif isinstance(initial_prompt, list):
            raise ValueError("TogetherAI does not support multiple prompts.")
        else:
            together_prompt = cast(str, initial_prompt)

        return together_prompt

    def transform_text_completion_request(
        self,
        model: str,
        messages: list[AllMessageValues] | list[OpenAITextCompletionUserMessage],
        optional_params: dict,
        headers: dict,
    ) -> dict:
        prompt: Final = self._transform_prompt(messages)
        return {

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Pass plain string content in messages instead of token IDs for together_ai models.
2. If you hold token IDs, decode them back to text first (tokenizer.decode(tokens)) before calling text_completion.
3. Route models that need token-level input to a provider that supports it, keeping together_ai models on string prompts.

Example fix

# before
resp = litellm.text_completion(
    model="together_ai/togethercomputer/LLaMA-2-7B-32K",
    messages=[{"role": "user", "content": {"tokens": [128000, 9707, 11]}}],
)

# after
from transformers import AutoTokenizer
enc = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf")
resp = litellm.text_completion(
    model="together_ai/togethercomputer/LLaMA-2-7B-32K",
    prompt=enc.decode([128000, 9707, 11]),
)

def is_token_payload(messages) -> bool:
    """Detect token-ID content that TogetherAI text completion rejects."""
    for m in messages:
        content = m.get("content") if isinstance(m, dict) else None
        if isinstance(content, dict) and (
            isinstance(content.get("tokens"), list)
            or isinstance(content.get("prompt_token_ids"), list)
        ):
            return True
    return False


assert not is_token_payload(messages), "decode tokens to text before together_ai calls"

def has_token_ids(value) -> bool:
    """Type guard: True when value is / contains integer token lists."""
    if isinstance(value, list) and value and all(isinstance(t, int) for t in value):
        return True
    if isinstance(value, list) and len(value) == 1 and isinstance(value[0], list):
        return all(isinstance(t, int) for t in value[0])
    return False

try:
    resp = litellm.text_completion(model="together_ai/...", prompt=payload)
except ValueError as e:
    if "does not support integers as input" in str(e):
        resp = litellm.text_completion(
            model="together_ai/...", prompt=tokenizer.decode(token_ids)
        )
    else:
        raise

When it happens

Trigger: Calling litellm.text_completion(model="together_ai/...", messages=[{'role':'user','content':{'tokens':[1234, 5678]}}]) or passing prompt_token_ids-style content blocks; any code path that hands LiteLLM tokenized prompts (e.g. caching layers that pre-tokenize) with a together_ai model.

Common situations: Migrating token-optimized pipelines from providers that accept token arrays (Anthropic/OpenAI content blocks, Vertex) to TogetherAI; prompt caching middleware that stores and replays token lists; test fixtures generated by a tokenizer.

Related errors

• TogetherAI does not support multiple prompts.
• TogetherAIException - {error_response['error']}
• TogetherAIException - {error_str}
• Batch record for /v1/completions is missing required `prompt
• TogetherAI does not support max_chunks_per_doc

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/d034ef84e9d0ade0. Report an issue: GitHub.