apache/seatunnel · error · SeaTunnelException

Failed to execute HTTP request to %s after %d attempts

Error message

Failed to execute HTTP request to %s after %d attempts

What it means

SeaTunnelException thrown by GravitinoClient.executeGetRequest after exhausting MAX_RETRY_ATTEMPTS on retryable failures (5xx, 408, 429, or IO errors, each with a backoff delay). Indicates persistent unavailability or overload of the Gravitino server.

Source

Thrown at seatunnel-api/src/main/java/org/apache/seatunnel/api/metalake/gravitino/GravitinoClient.java:175

                    }
                }
            } catch (IOException e) {
                if (attempt >= MAX_RETRY_ATTEMPTS) {
                    break;
                }
                // Exponential backoff delay before retry
                long delayMs = RETRY_DELAY_MS;
                log.warn(
                        "HTTP request to {} failed on attempt {}/{}, retrying in {}ms: {}",
                        url,
                        attempt,
                        MAX_RETRY_ATTEMPTS,
                        delayMs,
                        e.getMessage());
                sleepQuietly(delayMs);
            }
        }
        throw new SeaTunnelException(
                String.format(
                        "Failed to execute HTTP request to %s after %d attempts",
                        url, MAX_RETRY_ATTEMPTS));
    }

    /** 5xx and 408 and 429 will be retried */
    private boolean isRetryableHttpStatus(int httpStatus) {
        return httpStatus == HttpStatus.SC_INTERNAL_SERVER_ERROR
                || httpStatus == HttpStatus.SC_NOT_IMPLEMENTED
                || httpStatus == HttpStatus.SC_BAD_GATEWAY
                || httpStatus == HttpStatus.SC_SERVICE_UNAVAILABLE
                || httpStatus == HttpStatus.SC_GATEWAY_TIMEOUT
                || httpStatus == HttpStatus.SC_HTTP_VERSION_NOT_SUPPORTED
                || httpStatus == HttpStatus.SC_INSUFFICIENT_STORAGE
                || httpStatus == HttpStatus.SC_REQUEST_TIMEOUT
                || httpStatus == HttpStatus.SC_TOO_MANY_REQUESTS;
    }

View on GitHub (pinned to cf67b549a7)

Solutions

1. Check Gravitino server health and logs for the period of the failures.
2. Increase MAX_RETRY_ATTEMPTS/RETRY_DELAY_MS or make them configurable if the outage is expected to be transient.
3. Reduce concurrent request pressure or add client-side rate limiting to avoid 429s.
4. Fix underlying network connectivity/DNS issues between the SeaTunnel node and Gravitino.

Example fix

// before
JsonNode node = client.executeGetRequest(url); // throws after retries
// after
for (int i = 0; i < 3; i++) {
    try {
        JsonNode node = client.executeGetRequest(url);
        break;
    } catch (SeaTunnelException e) {
        Thread.sleep(30_000L); // tolerate longer Gravitino outages
    }
}

// preflight health probe with retry budget
for (int i = 0; i < 3; i++) {
    HttpURLConnection c = (HttpURLConnection) new URL(gravitinoBaseUri + "/api/metalakes").openConnection();
    c.setConnectTimeout(5000);
    if (c.getResponseCode() == 200) break;
    Thread.sleep(5000);
}

try {
    JsonNode node = client.executeGetRequest(url);
} catch (SeaTunnelException e) {
    if (e.getMessage().contains("after " + MAX_RETRY_ATTEMPTS + " attempts")) {
        LOG.error("Gravitino unavailable after {} retries: {}", MAX_RETRY_ATTEMPTS, url);
        // schedule job retry or fail with actionable message
    }
    throw e;
}

When it happens

Trigger: Calling any read path while the Gravitino server is down, continuously returning 5xx, rate-limiting (429), or timing out (408) for every attempt within the retry budget.

Common situations: Gravitino outage or restart loop; server overloaded by many concurrent metadata requests; aggressive rate limiting; long network partition between SeaTunnel and Gravitino.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• Stripe API rate limit reached, retry {}/{} after {} ms
• Failed to fetch metadata from Gravitino for metadata: %s
• fail get tableSchema:
• Failed to execute HTTP request to %s , http status code is %
• INSERT_DOC_ERROR

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/9855e5ec95088b38. Report an issue: GitHub.