BerriAI/litellm · error · ProxyRateLimitError

Batch rate limit exceeded for {descriptor.get('key', 'unknow

Error message

Batch rate limit exceeded for {descriptor.get('key', 'unknown')}: {descriptor.get('value', 'unknown')}. Batch contains {batch_usage.total_tokens} tokens but only {remaining_display} tokens remaining out of {current_limit} TPM limit. Limit resets at: {reset_time_formatted}

What it means

Error "Batch rate limit exceeded for {descriptor.get('key', 'unknown')}: {descriptor.get('value', 'unknown')}. Batch contains {batch_usage.total_tokens} tokens but only {remaining_display} tokens remaining out of {current_limit} TPM limit. Limit resets at: {reset_time_formatted}" thrown in BerriAI/litellm.

Source

Thrown at litellm/proxy/hooks/batch_rate_limiter.py:418

        current_limit: Final = status["current_limit"]

        if limit_type == "requests":
            detail = (
                f"Batch rate limit exceeded for {descriptor.get('key', 'unknown')}: {descriptor.get('value', 'unknown')}. "
                f"Batch contains {batch_usage.request_count} requests but only {remaining_display} requests remaining "
                f"out of {current_limit} RPM limit. "
                f"Limit resets at: {reset_time_formatted}"
            )
        else:  # tokens
            detail = (
                f"Batch rate limit exceeded for {descriptor.get('key', 'unknown')}: {descriptor.get('value', 'unknown')}. "
                f"Batch contains {batch_usage.total_tokens} tokens but only {remaining_display} tokens remaining "
                f"out of {current_limit} TPM limit. "
                f"Limit resets at: {reset_time_formatted}"
            )

        resolved_model, llm_provider = resolve_llm_provider_for_rate_limit(requested_model)
        raise ProxyRateLimitError(
            detail=detail,
            headers={
                "retry-after": str(window_size),
                "rate_limit_type": limit_type,
                "reset_at": reset_time_formatted,
            },
            category=RateLimitErrorCategory.LITELLM_BATCH_RATE_LIMIT,
            rate_limit_type=map_v3_rate_limit_type(limit_type),
            model=resolved_model,
            llm_provider=llm_provider,
        )

    async def _check_and_increment_batch_counters(
        self,
        user_api_key_dict: UserAPIKeyAuth,
        data: dict,
        batch_usage: BatchFileUsage,
        descriptors: list["RateLimitDescriptor"] | None = None,

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Reduce the total tokens in the batch to fit within the remaining TPM.
2. Wait until the limit reset time before resubmitting.
3. Request a higher TPM limit for the key or team.

When it happens

Trigger: Thrown at litellm/proxy/hooks/batch_rate_limiter.py:418 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/b042b79578ce5267. Report an issue: GitHub.