apache/seatunnel · warning

Airtable API rate limit reached, retry

Error message

Airtable API rate limit reached, retry {}/{} after {} ms

What it means

AirtableSinkWriter.sendWithRateLimitRetry, called from flush, detects HTTP 429 (TOO_MANY_REQUESTS) from the Airtable API and logs this warning before sleeping for an exponential backoff interval and retrying. Airtable enforces roughly 5 requests/second per base; when the sink exceeds that, responses come back with 429 and the writer retries up to rateLimitMaxRetries. If all retries are exhausted, the send fails.

Solutions

1. Increase the sink's rate-limit retry budget (rateLimitMaxRetries) so transient 429s are absorbed instead of failing the write.
2. Reduce write throughput: lower sink parallelism or increase batch size so fewer, larger requests are sent to Airtable.
3. Increase base backoff delay in calculateBackoffMillis and add jitter to avoid synchronized retries across writer subtasks.
4. Verify the API token is not shared with other services consuming the same base's rate quota.

Example fix

// before
AirtableSinkOptions: rateLimitMaxRetries = 3

// after
AirtableSinkOptions: rateLimitMaxRetries = 10  // absorb longer 429 storms
// plus reduce sink parallelism to 1 per base

try {
    writer.flush();
} catch (Exception e) {
    if (e.getMessage() != null && e.getMessage().contains("rate limit")) {
        LOG.warn("Airtable 429 after exhausting retries; backing off before resubmitting");
        Thread.sleep(60_000L);
        // resubmit or checkpoint-restart the job
    } else {
        throw e;
    }
}

When it happens

Trigger: Writing batches to Airtable faster than the per-base rate limit during flush(); a 429 response with retryCount < rateLimitMaxRetries triggers the warning and backoff sleep (calculateBackoffMillis(retryCount)).

Common situations: High-throughput sink parallelism hammering one Airtable base; small batch sizes causing many rapid API calls; shared rate budget exhausted by other applications using the same Airtable API token; transient Airtable-side throttling.

Related errors

• Airtable API rate limit reached, retry
• Failed to execute HTTP request to
• FLUSH_DATA_FAILED
• FLUSH_DATA_FAILED
• insert data failed, retry in smaller chunks

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/5285b7046c3aca59. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-http/connector-http-airtable/src/main/java/org/apache/seatunnel/connectors/seatunnel/airtable/sink/AirtableSinkWriter.java:138

        }

        return objectMapper.writeValueAsString(root);
    }

    private void sendWithRateLimitRetry(String body) throws IOException {
        int retryCount = 0;
        while (true) {
            waitForRequestSlot();
            try {
                HttpResponse response = httpClient.doPost(url, headers, body);
                if (HttpResponse.STATUS_OK == response.getCode()) {
                    return;
                }
                if (response.getCode() == STATUS_TOO_MANY_REQUESTS
                        && retryCount < rateLimitMaxRetries) {
                    retryCount++;
                    long backoffMillis = calculateBackoffMillis(retryCount);
                    log.warn(
                            "Airtable API rate limit reached, retry {}/{} after {} ms",
                            retryCount,
                            rateLimitMaxRetries,
                            backoffMillis);
                    try {
                        Thread.sleep(backoffMillis);
                    } catch (InterruptedException e) {
                        Thread.currentThread().interrupt();
                        throw new RuntimeException(e);
                    }
                    continue;
                }
                throw new IOException(
                        String.format(
                                "Airtable API request failed, status code:[%s], content:[%s]",
                                response.getCode(), response.getContent()));
            } catch (IOException e) {
                throw e;

View on GitHub (pinned to cf67b549a7)