{"record":{"id":"d034ef84e9d0ade0","repo":"BerriAI/litellm","slug":"togetherai-does-not-support-integers-as-input","errorCode":null,"errorMessage":"TogetherAI does not support integers as input","messagePattern":"TogetherAI does not support integers as input","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/together_ai/completion/transformation.py","lineNumber":33,"sourceCode":"    OpenAITextCompletionUserMessage,\n)\n\nfrom ...openai.completion.transformation import OpenAITextCompletionConfig\nfrom ...openai.completion.utils import _transform_prompt\n\n\nclass TogetherAITextCompletionConfig(OpenAITextCompletionConfig):\n    def _transform_prompt(\n        self,\n        messages: list[AllMessageValues] | list[OpenAITextCompletionUserMessage],\n    ) -> AllPromptValues:\n        \"\"\"\n        TogetherAI expects a string prompt.\n        \"\"\"\n        initial_prompt: Final[AllPromptValues] = _transform_prompt(messages)\n        ## TOGETHER AI SPECIFIC VALIDATION ##\n        if isinstance(initial_prompt, list) and is_tokens_or_list_of_tokens(value=initial_prompt):\n            raise ValueError(\"TogetherAI does not support integers as input\")\n        if isinstance(initial_prompt, list) and len(initial_prompt) == 1 and isinstance(initial_prompt[0], str):\n            together_prompt = initial_prompt[0]\n        elif isinstance(initial_prompt, list):\n            raise ValueError(\"TogetherAI does not support multiple prompts.\")\n        else:\n            together_prompt = cast(str, initial_prompt)\n\n        return together_prompt\n\n    def transform_text_completion_request(\n        self,\n        model: str,\n        messages: list[AllMessageValues] | list[OpenAITextCompletionUserMessage],\n        optional_params: dict,\n        headers: dict,\n    ) -> dict:\n        prompt: Final = self._transform_prompt(messages)\n        return {","sourceCodeStart":15,"sourceCodeEnd":51,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/together_ai/completion/transformation.py#L15-L51","documentation":"TogetherAI's text-completion endpoint accepts a plain string prompt only. LiteLLM first converts messages via _transform_prompt, and if the result is a list of token IDs (integers) — e.g. messages supplied with precomputed 'tokens'/'prompt_token_ids' — this validation raises immediately. TogetherAI has no token-list input mode, so the request can never be sent.","triggerScenarios":"Calling litellm.text_completion(model=\"together_ai/...\", messages=[{'role':'user','content':{'tokens':[1234, 5678]}}]) or passing prompt_token_ids-style content blocks; any code path that hands LiteLLM tokenized prompts (e.g. caching layers that pre-tokenize) with a together_ai model.","commonSituations":"Migrating token-optimized pipelines from providers that accept token arrays (Anthropic/OpenAI content blocks, Vertex) to TogetherAI; prompt caching middleware that stores and replays token lists; test fixtures generated by a tokenizer.","solutions":["Pass plain string content in messages instead of token IDs for together_ai models.","If you hold token IDs, decode them back to text first (tokenizer.decode(tokens)) before calling text_completion.","Route models that need token-level input to a provider that supports it, keeping together_ai models on string prompts."],"exampleFix":"# before\nresp = litellm.text_completion(\n    model=\"together_ai/togethercomputer/LLaMA-2-7B-32K\",\n    messages=[{\"role\": \"user\", \"content\": {\"tokens\": [128000, 9707, 11]}}],\n)\n\n# after\nfrom transformers import AutoTokenizer\nenc = AutoTokenizer.from_pretrained(\"meta-llama/Llama-2-7b-hf\")\nresp = litellm.text_completion(\n    model=\"together_ai/togethercomputer/LLaMA-2-7B-32K\",\n    prompt=enc.decode([128000, 9707, 11]),\n)","handlingStrategy":"type-guard","validationCode":"def is_token_payload(messages) -> bool:\n    \"\"\"Detect token-ID content that TogetherAI text completion rejects.\"\"\"\n    for m in messages:\n        content = m.get(\"content\") if isinstance(m, dict) else None\n        if isinstance(content, dict) and (\n            isinstance(content.get(\"tokens\"), list)\n            or isinstance(content.get(\"prompt_token_ids\"), list)\n        ):\n            return True\n    return False\n\n\nassert not is_token_payload(messages), \"decode tokens to text before together_ai calls\"","typeGuard":"def has_token_ids(value) -> bool:\n    \"\"\"Type guard: True when value is / contains integer token lists.\"\"\"\n    if isinstance(value, list) and value and all(isinstance(t, int) for t in value):\n        return True\n    if isinstance(value, list) and len(value) == 1 and isinstance(value[0], list):\n        return all(isinstance(t, int) for t in value[0])\n    return False","tryCatchPattern":"try:\n    resp = litellm.text_completion(model=\"together_ai/...\", prompt=payload)\nexcept ValueError as e:\n    if \"does not support integers as input\" in str(e):\n        resp = litellm.text_completion(\n            model=\"together_ai/...\", prompt=tokenizer.decode(token_ids)\n        )\n    else:\n        raise","preventionTips":["Standardize on string prompts at your service boundary and tokenize only inside provider adapters that accept it.","Add a pre-flight assertion that prompt is str for together_ai models.","Keep tokenizer handles alongside your cache so replayed token lists can always be decoded."],"tags":["together-ai","text-completion","prompt-format","token-ids","litellm"],"backgroundTag":"invalid-prompt-format","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}