{"record":{"id":"5285b7046c3aca59","repo":"apache/seatunnel","slug":"airtable-api-rate-limit-reached-retry-after","errorCode":null,"errorMessage":"Airtable API rate limit reached, retry {}/{} after {} ms","messagePattern":"Airtable API rate limit reached, retry (.+?)/(.+?) after (.+?) ms","errorType":"console","errorClass":null,"httpStatus":429,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-http/connector-http-airtable/src/main/java/org/apache/seatunnel/connectors/seatunnel/airtable/sink/AirtableSinkWriter.java","lineNumber":138,"sourceCode":"        }\n\n        return objectMapper.writeValueAsString(root);\n    }\n\n    private void sendWithRateLimitRetry(String body) throws IOException {\n        int retryCount = 0;\n        while (true) {\n            waitForRequestSlot();\n            try {\n                HttpResponse response = httpClient.doPost(url, headers, body);\n                if (HttpResponse.STATUS_OK == response.getCode()) {\n                    return;\n                }\n                if (response.getCode() == STATUS_TOO_MANY_REQUESTS\n                        && retryCount < rateLimitMaxRetries) {\n                    retryCount++;\n                    long backoffMillis = calculateBackoffMillis(retryCount);\n                    log.warn(\n                            \"Airtable API rate limit reached, retry {}/{} after {} ms\",\n                            retryCount,\n                            rateLimitMaxRetries,\n                            backoffMillis);\n                    try {\n                        Thread.sleep(backoffMillis);\n                    } catch (InterruptedException e) {\n                        Thread.currentThread().interrupt();\n                        throw new RuntimeException(e);\n                    }\n                    continue;\n                }\n                throw new IOException(\n                        String.format(\n                                \"Airtable API request failed, status code:[%s], content:[%s]\",\n                                response.getCode(), response.getContent()));\n            } catch (IOException e) {\n                throw e;","sourceCodeStart":120,"sourceCodeEnd":156,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-http/connector-http-airtable/src/main/java/org/apache/seatunnel/connectors/seatunnel/airtable/sink/AirtableSinkWriter.java#L120-L156","documentation":"AirtableSinkWriter.sendWithRateLimitRetry, called from flush, detects HTTP 429 (TOO_MANY_REQUESTS) from the Airtable API and logs this warning before sleeping for an exponential backoff interval and retrying. Airtable enforces roughly 5 requests/second per base; when the sink exceeds that, responses come back with 429 and the writer retries up to rateLimitMaxRetries. If all retries are exhausted, the send fails.","triggerScenarios":"Writing batches to Airtable faster than the per-base rate limit during flush(); a 429 response with retryCount < rateLimitMaxRetries triggers the warning and backoff sleep (calculateBackoffMillis(retryCount)).","commonSituations":"High-throughput sink parallelism hammering one Airtable base; small batch sizes causing many rapid API calls; shared rate budget exhausted by other applications using the same Airtable API token; transient Airtable-side throttling.","solutions":["Increase the sink's rate-limit retry budget (rateLimitMaxRetries) so transient 429s are absorbed instead of failing the write.","Reduce write throughput: lower sink parallelism or increase batch size so fewer, larger requests are sent to Airtable.","Increase base backoff delay in calculateBackoffMillis and add jitter to avoid synchronized retries across writer subtasks.","Verify the API token is not shared with other services consuming the same base's rate quota."],"exampleFix":"// before\nAirtableSinkOptions: rateLimitMaxRetries = 3\n\n// after\nAirtableSinkOptions: rateLimitMaxRetries = 10  // absorb longer 429 storms\n// plus reduce sink parallelism to 1 per base","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    writer.flush();\n} catch (Exception e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"rate limit\")) {\n        LOG.warn(\"Airtable 429 after exhausting retries; backing off before resubmitting\");\n        Thread.sleep(60_000L);\n        // resubmit or checkpoint-restart the job\n    } else {\n        throw e;\n    }\n}","preventionTips":["Keep sink parallelism at 1 per Airtable base to respect the ~5 req/sec per-base limit.","Increase batch size so fewer HTTP requests carry the same rows.","Set a generous rateLimitMaxRetries and use exponential backoff with jitter.","Avoid sharing one API token across multiple jobs/services hitting the same base."],"tags":["airtable","rate-limit","http-429","retry","sink"],"backgroundTag":"rate-limit-exceeded","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}