BerriAI/litellm · error · ValueError

Token counting request failed with status {response.status_c

Error message

Token counting request failed with status {response.status_code}: {error_text}

What it means

Raised after the async POST to the Vertex count-tokens endpoint returns a non-200 status. The handler surfaces the upstream status code and error body because the token-count request itself was rejected by Vertex.

Source

Thrown at litellm/llms/vertex_ai/vertex_ai_partner_models/count_tokens/handler.py:186

        # Get async HTTP client
        from litellm import LlmProviders

        async_client: Final = get_async_httpx_client(llm_provider=LlmProviders.VERTEX_AI)

        # Make the request
        # Note: Partner models (especially Claude) accept Anthropic Messages API format directly
        response: Final = await async_client.post(
            endpoint_url,
            headers=headers,
            json=request_data,
            timeout=30.0,
        )

        # Check for errors
        if response.status_code != 200:
            error_text: Final = response.text
            raise ValueError(f"Token counting request failed with status {response.status_code}: {error_text}")

        # Parse response
        result: Final = response.json()

        # Return token count
        # Vertex AI Anthropic returns: {"input_tokens": 123}
        return {
            "input_tokens": result.get("input_tokens", 0),
            "tokenizer_used": "vertex_ai_partner_models",
        }

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Read the status code and error text in the message for the upstream cause (auth, model access, quota).
2. Verify credentials, model availability in the region, and that the countTokens endpoint is reachable.

When it happens

Trigger: Thrown at litellm/llms/vertex_ai/vertex_ai_partner_models/count_tokens/handler.py:186 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/dba34be8e2ec838f. Report an issue: GitHub.