{"record":{"id":"27906adce46245b0","repo":"apache/iceberg","slug":"partial-commit-failed-for-task-will-retry","errorCode":null,"errorMessage":"Partial commit {} failed for task {}, will retry","messagePattern":"Partial commit (.+?) failed for task (.+?), will retry","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/channel/Coordinator.java","lineNumber":163,"sourceCode":"        commitState.addReady(envelope);\n        if (commitState.isCommitReady(totalPartitionCount)) {\n          commit(false);\n        }\n        return true;\n    }\n    return false;\n  }\n\n  private void commit(boolean partialCommit) {\n    try {\n      doCommit(partialCommit);\n      if (!partialCommit) {\n        consecutiveCommitFailures = 0;\n      }\n    } catch (RuntimeException e) {\n      if (partialCommit) {\n        partialCommitFailures.incrementAndGet();\n        LOG.warn(\n            \"Partial commit {} failed for task {}, will retry\",\n            commitState.currentCommitId(),\n            taskId,\n            e);\n        return;\n      }\n\n      if (!(e instanceof CommitFailedException)) {\n        // CommitStateUnknownException, ValidationException, ForbiddenException,\n        // NPE, anything else -- not retryable, terminate immediately\n        throw e;\n      }\n\n      consecutiveCommitFailures++;\n      if (consecutiveCommitFailures >= config.commitMaxConsecutiveFailures()) {\n        LOG.error(\n            \"Commit {} failed for task {} ({} consecutive failures, terminating)\",\n            commitState.currentCommitId(),","sourceCodeStart":145,"sourceCodeEnd":181,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/channel/Coordinator.java#L145-L181","documentation":"During a partial commit (some tasks committed, retrying the remainder), a RuntimeException was caught. Instead of failing, the coordinator increments partialCommitFailures and logs this warning, because a partial commit will be retried. The exception is deliberately swallowed to allow the retry loop to continue.","triggerScenarios":"commitToTable/doCommit throws for a partial commit — e.g. catalog.loadTable or table commit contention (CommitFailedException), transient broker/catalog errors — while other tasks already committed their part.","commonSituations":"Concurrent Iceberg table commits causing retries; REST catalog 5xx responses; S3/Glue throttling; a table locked by another process during a multi-task commit.","solutions":["Let the retry mechanism run; partial commits retry automatically and no manual intervention is needed for transient errors.","Check the attached exception 'e' for the root cause (e.g. CommitFailedException vs catalog outage) and fix that underlying issue.","Reduce concurrent writers to the target tables or increase retry tolerances to lower contention.","If failures escalate until the commit ultimately fails, investigate the specific table's commit conflicts (branch state, competing jobs)."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  coordinator.commit(...);\n} catch (RuntimeException e) {\n  // partial commits retry automatically; inspect 'e' for the root cause\n  // (CommitFailedException, catalog 5xx, throttling) and fix underlying issue\n}","preventionTips":["Reduce concurrent Iceberg writers on connector target tables.","Keep catalog endpoints healthy and credentials fresh.","Watch partialCommitFailures growth to detect persistent contention.","Schedule table maintenance (compaction, expiry) away from commit windows."],"tags":["kafka-connect","commit","retry","iceberg"],"backgroundTag":"partial-commit-failure","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}