apache/seatunnel · critical · RuntimeException
Failed to write items to table after retries
Error message
Failed to write %d items to table %s after %d retries
What it means
DynamoDbSinkClient.flushWithRetry retries failed writes up to maxRetries; afterward, any still-pending (unprocessed or failed) WriteRequests cause this RuntimeException reporting the item count, table name, and retry count. It means the batch write permanently failed after exhausting retries.
Solutions
- Increase DynamoDB write capacity (or use on-demand mode) / reduce sink parallelism and batch size to stay under throttling limits.
- Increase maxRetries and backoff in the sink configuration for transient throttling.
- Filter or split oversized items (>400KB) and fix invalid attribute values before writing.
- Check the log lines just before the exception — they list the count and table; correlate with DynamoDB throttling metrics (ThrottledRequests, WriteThrottleEvents).
Example fix
// before maxRetries = 3; batchSize = 25 // after: more retries, smaller batches maxRetries = 10; batchSize = 10
Defensive patterns
Strategy: retry
Validate before calling
for (WriteRequest r : requests) {
if (r instanceof WriteRequest && estimatedSize(r) > 400 * 1024) throw new IllegalArgumentException("Item exceeds 400KB DynamoDB limit");
} Type guard
null
Try / catch
try { client.flushWithRetry(); } catch (RuntimeException e) { log.error("DynamoDB flush failed: {}", e.getMessage()); alertAndCheckpointReplay(); } Prevention
- Size the table capacity (or on-demand) for peak write throughput; watch WriteThrottleEvents metrics.
- Keep batch size modest and items under 400KB.
- Increase maxRetries with exponential backoff for throttling-prone workloads.
When it happens
Trigger: batchWriteItem repeatedly returns unprocessed items or errors (provisioned throughput exceeded, item size > 400KB, invalid items) across all maxRetries attempts, leaving pendingRequests non-empty at the end of flushWithRetry.
Common situations: DynamoDB table under-provisioned or in on-demand throttle due to hot partition keys; single items exceeding the 400KB DynamoDB limit; duplicate keys or invalid attribute values in the batch; sustained write rate above table capacity during bulk load.
Related errors
- Interrupted during retry
- COMMON-17
- FLUSH_DATA_FAILED
- Graph operation failed on attempt
- ACKNOWLEDGE_FAILED
AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10).
Data as JSON: /api/errors/fa3c8e549ef8dc5c.
Report an issue: GitHub.
Appendix: source
Thrown at seatunnel-connectors-v2/connector-amazondynamodb/src/main/java/org/apache/seatunnel/connectors/seatunnel/amazondynamodb/sink/DynamoDbSinkClient.java:195
delay);
try {
Thread.sleep(delay);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RuntimeException("Interrupted during retry", e);
}
}
}
if (!pendingRequests.isEmpty()) {
log.error(
"Failed to write {} items to table '{}' after {} retries",
pendingRequests.size(),
tableName,
maxRetries);
throw new RuntimeException(
String.format(
"Failed to write %d items to table %s after %d retries",
pendingRequests.size(), tableName, maxRetries));
}
}
}
View on GitHub (pinned to cf67b549a7)