{"record":{"id":"fc5ed5f8de2360d3","repo":"BerriAI/litellm","slug":"unexpected-error-during-token-counting-e","errorCode":null,"errorMessage":"Unexpected error during token counting: {e}","messagePattern":"Unexpected error during token counting: (.+?)","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"litellm/llms/gemini/count_tokens/handler.py","lineNumber":162,"sourceCode":"\n            # Parse response\n            result: Final = response.json()\n            return result\n\n        except httpx.HTTPStatusError as e:\n            error_msg = f\"Google Gen AI Studio API error: {e.response.status_code} - {e.response.text}\"\n            raise litellm.APIError(\n                message=error_msg,\n                llm_provider=\"gemini\",\n                model=model,\n                status_code=e.response.status_code,\n            ) from e\n        except httpx.RequestError as e:\n            error_msg = f\"Request to Google Gen AI Studio failed: {e}\"\n            raise litellm.APIConnectionError(message=error_msg, llm_provider=\"gemini\", model=model) from e\n        except Exception as e:\n            error_msg = f\"Unexpected error during token counting: {e}\"\n            raise Exception(error_msg) from e\n","sourceCodeStart":144,"sourceCodeEnd":163,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/gemini/count_tokens/handler.py#L144-L163","documentation":"The Gemini count-tokens handler wraps the HTTP call in a three-branch exception ladder: API errors become litellm.APIError, network failures become APIConnectionError, and this final except Exception re-raises any other failure (JSON decode errors, bugs in response parsing, KeyErrors on an unexpected payload) as a plain Exception with the 'Unexpected error during token counting' prefix, chaining the original cause via 'from e'.","triggerScenarios":"Calling the Gemini token-counting endpoint and hitting a response that is 2xx but not the expected JSON shape (missing fields), a body that fails to parse, or any non-HTTP, non-httpx exception raised during transformation of the response.","commonSituations":"Google changes the countTokens response schema in vN; a proxy returns an HTML error page with status 200; partial responses on flaky connections; version skew between litellm's parser and the API it calls.","solutions":["Inspect the chained exception (__cause__) — it names the real failure (KeyError, JSONDecodeError, etc.) and usually the offending field.","Upgrade litellm to the latest patch release so the countTokens parser matches the current Gemini schema.","If a proxy is in the path, confirm it passes JSON through unmodified (content-type and body)."],"exampleFix":"# before\ntry:\n    n = litellm.token_counter(model=\"gemini/gemini-2.0-flash\", messages=msg_list)\nexcept Exception as e:\n    print(e)  # opaque \"Unexpected error...\"\n\n# after\ntry:\n    n = litellm.token_counter(model=\"gemini/gemini-2.0-flash\", messages=msg_list)\nexcept Exception as e:\n    logging.exception(\"count-tokens failed\")  # logs chained __cause__\n    n = litellm.token_counter(model=\"gpt-4o-mini\", messages=msg_list)  # fallback estimate or another model","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    n = litellm.token_counter(model=\"gemini/gemini-2.0-flash\", messages=msgs)\nexcept Exception as e:\n    if \"Unexpected error during token counting\" in str(e):\n        logging.warning(\"Gemini countTokens failed (%r); estimating locally\", e.__cause__)\n        n = estimate_tokens_locally(msgs)  # e.g. len(text) // 4\n    else:\n        raise","preventionTips":["Token counts used for budgeting do not need exactness — keep a local estimator as a fallback.","Log e.__cause__ (the chained exception) to identify schema-drift root causes quickly.","Keep litellm updated when Google revises the countTokens response shape."],"tags":["gemini","token-counting","response-parsing","exception-handling"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}