{"record":{"id":"acbcc711c619dc1c","repo":"invoke-ai/InvokeAI","slug":"rate-limit-exceeded-after-all-retries","errorCode":null,"errorMessage":"Rate limit exceeded after all retries","messagePattern":"Rate limit exceeded after all retries","errorType":"exception","errorClass":"ExternalProviderRateLimitError","httpStatus":null,"severity":"error","filePath":"invokeai/app/services/external_generation/external_generation_default.py","lineNumber":89,"sourceCode":"    def _generate_with_retry(\n        self, provider: ExternalProvider, request: ExternalGenerationRequest\n    ) -> ExternalGenerationResult:\n        for attempt in range(self._MAX_RETRIES):\n            try:\n                return provider.generate(request)\n            except ExternalProviderRateLimitError as exc:\n                if attempt == self._MAX_RETRIES - 1:\n                    raise\n                delay = min(exc.retry_after or self._DEFAULT_RETRY_DELAY, self._MAX_RETRY_DELAY)\n                self._logger.warning(\n                    \"Rate limited by %s (attempt %d/%d), retrying in %.0fs\",\n                    request.model.provider_id,\n                    attempt + 1,\n                    self._MAX_RETRIES,\n                    delay,\n                )\n                time.sleep(delay)\n        raise ExternalProviderRateLimitError(\"Rate limit exceeded after all retries\")\n\n    def get_provider_statuses(self) -> dict[str, ExternalProviderStatus]:\n        return {provider_id: provider.get_status() for provider_id, provider in self._providers.items()}\n\n    def _validate_request(self, request: ExternalGenerationRequest) -> None:\n        capabilities = request.model.capabilities\n\n        self._logger.debug(\n            \"Validating external request provider=%s model=%s mode=%s supported=%s\",\n            request.model.provider_id,\n            request.model.provider_model_id,\n            request.mode,\n            capabilities.modes,\n        )\n\n        if request.mode not in capabilities.modes:\n            raise ExternalProviderCapabilityError(f\"Mode '{request.mode}' is not supported by {request.model.name}\")\n","sourceCodeStart":71,"sourceCodeEnd":107,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/app/services/external_generation/external_generation_default.py#L71-L107","documentation":"ExternalProviderRateLimitError raised by _generate_with_retry after the request was retried _MAX_RETRIES times and the external provider kept returning rate-limit (HTTP 429) responses. Indicates sustained throttling by the upstream provider, not a client bug.","triggerScenarios":"generate() -> _generate_with_retry exhausts all retries with exponential backoff because every call to the external provider API returns a rate-limit response.","commonSituations":"Shared/free-tier API keys hitting provider quotas; batch jobs issuing many generation requests back-to-back; multiple users on the same API key; provider-side capacity limits during peak hours.","solutions":["Wait and retry later once the provider's rate-limit window resets","Upgrade the provider API key tier or raise its quota","Reduce request rate: add client-side throttling/queuing between generation requests","Switch to a different configured provider for this workload","Increase _MAX_RETRIES / backoff delay if sustained bursts are expected"],"exampleFix":"// before: tight loop of generate() calls\nfor prompt in prompts:\n    service.generate(make_request(prompt))\n\n// after: throttle client-side\nfor prompt in prompts:\n    service.generate(make_request(prompt))\n    time.sleep(5)  # stay under provider RPM","handlingStrategy":"retry","validationCode":"# No pre-call check possible for server-side rate limits; pace calls proactively\nMIN_INTERVAL = 5.0\nlast = [0.0]\ndef pace():\n    wait = MIN_INTERVAL - (time.monotonic() - last[0])\n    if wait > 0: time.sleep(wait)\n    last[0] = time.monotonic()","typeGuard":"None","tryCatchPattern":"try:\n    result = service.generate(request)\nexcept ExternalProviderRateLimitError:\n    time.sleep(60)  # wait out the provider's rate window\n    result = service.generate(request)  # or fail over to another provider","preventionTips":["Throttle/queue outgoing generation requests under the provider's RPM/TPM limits","Use a paid/higher-tier API key for batch workloads","Spread long batch jobs with delays; avoid bursts from parallel workers sharing one key","Monitor provider usage dashboards and alert before hitting quotas"],"tags":["rate-limit","retry","external-api","throttling"],"backgroundTag":"rate-limit-exceeded","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}