{"record":{"id":"cdd1906ae4ceec23","repo":"BerriAI/litellm","slug":"togetherai-does-not-support-max-chunks-per-doc","errorCode":null,"errorMessage":"TogetherAI does not support max_chunks_per_doc","messagePattern":"TogetherAI does not support max_chunks_per_doc","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/together_ai/rerank/handler.py","lineNumber":46,"sourceCode":"        return_documents: bool | None = True,\n        max_chunks_per_doc: int | None = None,\n        _is_async: bool | None = False,\n    ) -> RerankResponse:\n        client: Final = _get_httpx_client()\n\n        request_data: Final = RerankRequest(\n            model=model,\n            query=query,\n            top_n=top_n,\n            documents=documents,\n            rank_fields=rank_fields,\n            return_documents=return_documents,\n        )\n\n        # exclude None values from request_data\n        request_data_dict: Final = request_data.dict(exclude_none=True)\n        if max_chunks_per_doc is not None:\n            raise ValueError(\"TogetherAI does not support max_chunks_per_doc\")\n\n        if _is_async:\n            return self.async_rerank(request_data_dict, api_key)  # Call async method\n\n        response: Final = client.post(\n            \"https://api.together.xyz/v1/rerank\",\n            headers={\n                \"accept\": \"application/json\",\n                \"content-type\": \"application/json\",\n                \"authorization\": f\"Bearer {api_key}\",\n            },\n            json=request_data_dict,\n        )\n\n        if response.status_code != 200:\n            raise Exception(response.text)\n\n        _json_response: Final = response.json()","sourceCodeStart":28,"sourceCodeEnd":64,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/together_ai/rerank/handler.py#L28-L64","documentation":"LiteLLM's TogetherAI rerank handler builds a RerankRequest and explicitly rejects max_chunks_per_doc before sending anything to https://api.together.xyz/v1/rerank, because Together's rerank API has no such option. If the caller passes max_chunks_per_doc (even as None-adjacent optional config) and it is not None, a ValueError is raised client-side.","triggerScenarios":"Calling litellm.rerank(model=\"together_ai/rerank-english-v2.0\", query=..., documents=..., max_chunks_per_doc=...) with any non-None value; generic rerank wrappers that forward the full Cohere-style parameter set to every provider.","commonSituations":"Sharing one rerank call signature across Cohere and TogetherAI — Cohere accepts max_chunks_per_doc, Together does not; copying sample code from a Cohere rerank guide and switching only the model string; config-driven rerank defaults that set the parameter for all deployments.","solutions":["Remove max_chunks_per_doc from the rerank call when model starts with together_ai/.","Branch your wrapper: pass max_chunks_per_doc only for providers that support it (e.g. Cohere).","Chunk long documents yourself before calling TogetherAI rerank if you need chunk-level control."],"exampleFix":"# before\nresp = litellm.rerank(\n    model=\"together_ai/rerank-english-v2.0\",\n    query=\"What is the capital of the US?\",\n    documents=[\"Washington, D.C. ...\", \"Paris is ...\"],\n    max_chunks_per_doc=16,\n)\n\n# after\nresp = litellm.rerank(\n    model=\"together_ai/rerank-english-v2.0\",\n    query=\"What is the capital of the US?\",\n    documents=[\"Washington, D.C. ...\", \"Paris is ...\"],\n)","handlingStrategy":"validation","validationCode":"TOGETHER_UNSUPPORTED_RERANK_PARAMS = {\"max_chunks_per_doc\"}\n\n\ndef build_rerank_kwargs(model: str, **kwargs) -> dict:\n    \"\"\"Strip params the target rerank provider cannot take.\"\"\"\n    if model.startswith(\"together_ai/\"):\n        blocked = TOGETHER_UNSUPPORTED_RERANK_PARAMS & set(kwargs)\n        if blocked:\n            raise ValueError(f\"together_ai rerank does not accept: {sorted(blocked)}\")\n    return kwargs","typeGuard":null,"tryCatchPattern":"try:\n    resp = litellm.rerank(model=\"together_ai/rerank-english-v2.0\", query=q, documents=docs, **params)\nexcept ValueError as e:\n    if \"does not support max_chunks_per_doc\" in str(e):\n        params.pop(\"max_chunks_per_doc\", None)\n        resp = litellm.rerank(model=\"together_ai/rerank-english-v2.0\", query=q, documents=docs, **params)\n    else:\n        raise","preventionTips":["Keep a per-provider allowlist of rerank parameters in your retrieval layer.","Don't copy Cohere rerank examples verbatim when switching model strings.","Validate kwargs against the allowlist at request-build time, not after the call."],"tags":["together-ai","rerank","parameter-validation","litellm"],"backgroundTag":"unsupported-request-parameter","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}