BerriAI/litellm · error · Exception

Unexpected error during token counting: {e}

Error message

Unexpected error during token counting: {e}

What it means

The Gemini count-tokens handler wraps the HTTP call in a three-branch exception ladder: API errors become litellm.APIError, network failures become APIConnectionError, and this final except Exception re-raises any other failure (JSON decode errors, bugs in response parsing, KeyErrors on an unexpected payload) as a plain Exception with the 'Unexpected error during token counting' prefix, chaining the original cause via 'from e'.

Source

Thrown at litellm/llms/gemini/count_tokens/handler.py:162

            # Parse response
            result: Final = response.json()
            return result

        except httpx.HTTPStatusError as e:
            error_msg = f"Google Gen AI Studio API error: {e.response.status_code} - {e.response.text}"
            raise litellm.APIError(
                message=error_msg,
                llm_provider="gemini",
                model=model,
                status_code=e.response.status_code,
            ) from e
        except httpx.RequestError as e:
            error_msg = f"Request to Google Gen AI Studio failed: {e}"
            raise litellm.APIConnectionError(message=error_msg, llm_provider="gemini", model=model) from e
        except Exception as e:
            error_msg = f"Unexpected error during token counting: {e}"
            raise Exception(error_msg) from e

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Inspect the chained exception (__cause__) — it names the real failure (KeyError, JSONDecodeError, etc.) and usually the offending field.
2. Upgrade litellm to the latest patch release so the countTokens parser matches the current Gemini schema.
3. If a proxy is in the path, confirm it passes JSON through unmodified (content-type and body).

Example fix

# before
try:
    n = litellm.token_counter(model="gemini/gemini-2.0-flash", messages=msg_list)
except Exception as e:
    print(e)  # opaque "Unexpected error..."

# after
try:
    n = litellm.token_counter(model="gemini/gemini-2.0-flash", messages=msg_list)
except Exception as e:
    logging.exception("count-tokens failed")  # logs chained __cause__
    n = litellm.token_counter(model="gpt-4o-mini", messages=msg_list)  # fallback estimate or another model

try:
    n = litellm.token_counter(model="gemini/gemini-2.0-flash", messages=msgs)
except Exception as e:
    if "Unexpected error during token counting" in str(e):
        logging.warning("Gemini countTokens failed (%r); estimating locally", e.__cause__)
        n = estimate_tokens_locally(msgs)  # e.g. len(text) // 4
    else:
        raise

When it happens

Trigger: Calling the Gemini token-counting endpoint and hitting a response that is 2xx but not the expected JSON shape (missing fields), a body that fails to parse, or any non-HTTP, non-httpx exception raised during transformation of the response.

Common situations: Google changes the countTokens response schema in vN; a proxy returns an HTML error page with status 200; partial responses on flaky connections; version skew between litellm's parser and the API it calls.

Related errors

• Error parsing file upload response: {e}
• Failed to parse Braintrust API response: {str(e)}
• Error apply_db_fixes: {str(e)}
• No audio part found in the response
• function_call missing. Received tool call with 'type': 'func

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/fc5ed5f8de2360d3. Report an issue: GitHub.