{"record":{"id":"a1827b870dc9e49a","repo":"apache/beam","slug":"request-failed-after-exhausting-retries-max-retries","errorCode":null,"errorMessage":"Request failed after exhausting retries. Max retries: {maxRetries}, ","messagePattern":"Request failed after exhausting retries\\. Max retries: (.+?), ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/ml/inference/remote/src/main/java/org/apache/beam/sdk/ml/inference/remote/RetryHandler.java","lineNumber":102,"sourceCode":"    int attempt = 0;\n\n    while (true) {\n      try {\n        return request.call();\n\n      } catch (Exception e) {\n        lastException = e;\n\n        if (retryFilter != null && !retryFilter.shouldRetry(e)) {\n          LOG.warn(\"Exception not eligible for retry. Failing immediately.\", e);\n          throw e;\n        }\n\n        long backoffMillis = backoff.nextBackOffMillis();\n\n        if (backoffMillis == BackOff.STOP) {\n          LOG.error(\"Request failed after {} retry attempts.\", attempt);\n          throw new RuntimeException(\n              \"Request failed after exhausting retries. \" + \"Max retries: \" + maxRetries + \", \",\n              lastException);\n        }\n\n        attempt++;\n        LOG.warn(\"Retry request attempt {} failed. Retrying in {} ms\", attempt, backoffMillis, e);\n\n        sleeper.sleep(backoffMillis);\n      }\n    }\n  }\n\n  @FunctionalInterface\n  public interface RetryableRequest<T> {\n\n    T call() throws Exception;\n  }\n}","sourceCodeStart":84,"sourceCodeEnd":120,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/ml/inference/remote/src/main/java/org/apache/beam/sdk/ml/inference/remote/RetryHandler.java#L84-L120","documentation":"RetryHandler.execute() retries a request with exponential backoff until the BackOff signals STOP. When retries are exhausted without success, it throws a RuntimeException carrying the max-retry count and the last exception as cause, after logging the failure.","triggerScenarios":"execute()/executeWithT() called against an endpoint that keeps failing (connection errors, 5xx, timeouts) for more than maxRetries consecutive attempts with the configured backoff.","commonSituations":"Remote inference endpoint down or misconfigured URL, persistent 429/5xx from an overloaded service, DNS/firewall issues on workers, and maxRetries set too low for a flaky network.","solutions":["Inspect lastException (getCause()) for the underlying network/HTTP failure.","Increase maxRetries and/or the backoff period for transient upstream issues.","Verify the endpoint URL, DNS, and network egress from the Beam workers.","Address server-side causes: rate limits (back off more), service health, capacity."],"exampleFix":"// before\nRetryHandler rh = new RetryHandler(options, 2, backoff);\n// after\nRetryHandler rh = new RetryHandler(options, 10, BackOff.EXPONENTIAL_BACKOFF.withMaxRetries(10).withInitialBackoff(Duration.millis(500)));","handlingStrategy":"retry","validationCode":"if (!endpointReachable(url, timeoutMs)) { throw new IllegalStateException(\"remote inference endpoint unreachable: \" + url); }","typeGuard":null,"tryCatchPattern":"try { result = retryHandler.executeWithT(() -> callEndpoint()); } catch (RuntimeException e) { if (e.getMessage().contains(\"exhausting retries\")) { /* alert + inspect e.getCause(); consider circuit-breaker */ } throw e; }","preventionTips":["Size maxRetries/backoff to the endpoint's expected recovery time","Monitor endpoint health and rate limits","Verify worker network egress/DNS before launching","Use idempotent requests so retries are safe"],"tags":["java","retry","network","remote-inference"],"backgroundTag":"request-timeout","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}