{"record":{"id":"a4400a76b18f897a","repo":"apache/seatunnel","slug":"airtable-api-rate-limit-reached-retry-after-a4400a","errorCode":null,"errorMessage":"Airtable API rate limit reached, retry {}/{} after {} ms","messagePattern":"Airtable API rate limit reached, retry (.+?)/(.+?) after (.+?) ms","errorType":"console","errorClass":null,"httpStatus":429,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-http/connector-http-airtable/src/main/java/org/apache/seatunnel/connectors/seatunnel/airtable/source/AirtableSourceReader.java","lineNumber":72,"sourceCode":"            int rateLimitBackoffMs,\n            int rateLimitMaxRetries) {\n        super(httpParameter, context, deserializationSchema, jsonField, contentJson, pageInfo);\n        this.requestIntervalMs = Math.max(0, requestIntervalMs);\n        this.rateLimitBackoffMs = Math.max(0, rateLimitBackoffMs);\n        this.rateLimitMaxRetries = Math.max(0, rateLimitMaxRetries);\n    }\n\n    @Override\n    protected HttpResponse executeRequest() throws Exception {\n        int retryCount = 0;\n        while (true) {\n            waitForRequestSlot();\n            HttpResponse response = doExecuteRequest();\n            if (response.getCode() == STATUS_TOO_MANY_REQUESTS\n                    && retryCount < rateLimitMaxRetries) {\n                retryCount += 1;\n                long backoffMillis = calculateBackoffMillis(retryCount);\n                log.warn(\n                        \"Airtable API rate limit reached, retry {}/{} after {} ms\",\n                        retryCount,\n                        rateLimitMaxRetries,\n                        backoffMillis);\n                try {\n                    Thread.sleep(backoffMillis);\n                } catch (InterruptedException e) {\n                    Thread.currentThread().interrupt();\n                    throw new RuntimeException(e);\n                }\n                continue;\n            }\n            return response;\n        }\n    }\n\n    private HttpResponse doExecuteRequest() throws Exception {\n        return httpClient.execute(","sourceCodeStart":54,"sourceCodeEnd":90,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-http/connector-http-airtable/src/main/java/org/apache/seatunnel/connectors/seatunnel/airtable/source/AirtableSourceReader.java#L54-L90","documentation":"AirtableSourceReader.executeRequest (used to build the HTTP response) detects HTTP 429 from the Airtable API after waitForRequestSlot() and doExecuteRequest(), logs this warning, and sleeps for a calculated backoff before retrying. Airtable's read API is rate limited (~5 req/sec per base); the reader retries up to rateLimitMaxRetries before giving up, so this warning indicates throttling that is being actively retried.","triggerScenarios":"Polling/reading from Airtable faster than the per-base limit during executeRequest(); a 429 status with retryCount < rateLimitMaxRetries triggers the warning, backoff sleep, and retry loop.","commonSituations":"Many source reader subtasks polling the same base and sharing the rate budget; frequent incremental polling intervals shorter than the limit allows; other applications using the same Airtable token exhausting the quota; large tables paginated with many rapid requests.","solutions":["Increase rateLimitMaxRetries so the reader can ride out sustained 429 responses.","Reduce reader parallelism or slow the polling interval so requests stay under ~5 req/sec per base.","Increase the base backoff delay and add jitter in calculateBackoffMillis to avoid thundering-herd retries.","Check whether other consumers share the same Airtable API key/base and are consuming the quota."],"exampleFix":"// before\nsource.polling.interval = 1000; parallelism = 4;\n\n// after\nsource.polling.interval = 5000; parallelism = 1; // stay under Airtable 5 req/sec limit","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    HttpResponse resp = reader.executeRequest();\n    if (resp.getCode() == 429) {\n        // retries were exhausted; wait a full reset window before retrying\n        Thread.sleep(60_000L);\n    }\n} catch (Exception e) {\n    LOG.warn(\"Airtable read throttled beyond retry budget\", e);\n    // rely on source restart/checkpoint recovery to re-poll\n}","preventionTips":["Lower polling frequency below Airtable's ~5 req/sec per-base limit.","Use a single reader subtask per base; Airtable limits are per base, not per task.","Configure rateLimitMaxRetries high enough to survive sustained throttling windows.","Stagger polling intervals and add backoff jitter when multiple jobs read the same base."],"tags":["airtable","rate-limit","http-429","retry","source"],"backgroundTag":"rate-limit-exceeded","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}