apache/seatunnel · warning

Airtable API rate limit reached, retry

Error message

Airtable API rate limit reached, retry {}/{} after {} ms

What it means

AirtableSourceReader.executeRequest (used to build the HTTP response) detects HTTP 429 from the Airtable API after waitForRequestSlot() and doExecuteRequest(), logs this warning, and sleeps for a calculated backoff before retrying. Airtable's read API is rate limited (~5 req/sec per base); the reader retries up to rateLimitMaxRetries before giving up, so this warning indicates throttling that is being actively retried.

Solutions

1. Increase rateLimitMaxRetries so the reader can ride out sustained 429 responses.
2. Reduce reader parallelism or slow the polling interval so requests stay under ~5 req/sec per base.
3. Increase the base backoff delay and add jitter in calculateBackoffMillis to avoid thundering-herd retries.
4. Check whether other consumers share the same Airtable API key/base and are consuming the quota.

Example fix

// before
source.polling.interval = 1000; parallelism = 4;

// after
source.polling.interval = 5000; parallelism = 1; // stay under Airtable 5 req/sec limit

try {
    HttpResponse resp = reader.executeRequest();
    if (resp.getCode() == 429) {
        // retries were exhausted; wait a full reset window before retrying
        Thread.sleep(60_000L);
    }
} catch (Exception e) {
    LOG.warn("Airtable read throttled beyond retry budget", e);
    // rely on source restart/checkpoint recovery to re-poll
}

When it happens

Trigger: Polling/reading from Airtable faster than the per-base limit during executeRequest(); a 429 status with retryCount < rateLimitMaxRetries triggers the warning, backoff sleep, and retry loop.

Common situations: Many source reader subtasks polling the same base and sharing the rate budget; frequent incremental polling intervals shorter than the limit allows; other applications using the same Airtable token exhausting the quota; large tables paginated with many rapid requests.

Related errors

• Airtable API rate limit reached, retry
• Edge socket receiver loop exception, retrying
• Failed to execute HTTP request to
• insert data failed, retry in smaller chunks
• Stripe API rate limit reached, retry

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/a4400a76b18f897a. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-http/connector-http-airtable/src/main/java/org/apache/seatunnel/connectors/seatunnel/airtable/source/AirtableSourceReader.java:72

            int rateLimitBackoffMs,
            int rateLimitMaxRetries) {
        super(httpParameter, context, deserializationSchema, jsonField, contentJson, pageInfo);
        this.requestIntervalMs = Math.max(0, requestIntervalMs);
        this.rateLimitBackoffMs = Math.max(0, rateLimitBackoffMs);
        this.rateLimitMaxRetries = Math.max(0, rateLimitMaxRetries);
    }

    @Override
    protected HttpResponse executeRequest() throws Exception {
        int retryCount = 0;
        while (true) {
            waitForRequestSlot();
            HttpResponse response = doExecuteRequest();
            if (response.getCode() == STATUS_TOO_MANY_REQUESTS
                    && retryCount < rateLimitMaxRetries) {
                retryCount += 1;
                long backoffMillis = calculateBackoffMillis(retryCount);
                log.warn(
                        "Airtable API rate limit reached, retry {}/{} after {} ms",
                        retryCount,
                        rateLimitMaxRetries,
                        backoffMillis);
                try {
                    Thread.sleep(backoffMillis);
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                    throw new RuntimeException(e);
                }
                continue;
            }
            return response;
        }
    }

    private HttpResponse doExecuteRequest() throws Exception {
        return httpClient.execute(

View on GitHub (pinned to cf67b549a7)