apache/seatunnel · error · RuntimeException

Failed to flush data in prepareCommit

Error message

Failed to flush data in prepareCommit

What it means

HttpSinkWriter.prepareCommit flushes buffered rows only in arrayMode; if that flush throws an IOException it is rethrown as RuntimeException 'Failed to flush data in prepareCommit', failing the checkpoint. This ensures batched HTTP writes are committed or the task fails before acknowledgment.

Source

Thrown at seatunnel-connectors-v2/connector-http/connector-http-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/http/sink/HttpSinkWriter.java:170

    }

    @Override
    public void close() throws IOException {
        if (arrayMode) {
            flush();
        }
        if (Objects.nonNull(httpClient)) {
            httpClient.close();
        }
    }

    @Override
    public Optional<Void> prepareCommit() {
        if (arrayMode) {
            try {
                flush();
            } catch (IOException e) {
                throw new RuntimeException("Failed to flush data in prepareCommit", e);
            }
        }
        return Optional.empty();
    }

    @VisibleForTesting
    protected HttpClientProvider createHttpClient(HttpParameter httpParameter) {
        return new HttpClientProvider(httpParameter);
    }
}

View on GitHub (pinned to cf67b549a7)

Solutions

1. Check the cause for the underlying HTTP failure (status code / connection error).
2. Verify sink URL, headers, and that the endpoint accepts JSON arrays in array mode.
3. Reduce flush batch size or checkpoint interval to lower commit-time pressure.
4. Add retry/health handling on the receiving service; restart from the last checkpoint after fixing.

// Pre-check the endpoint accepts the payload
curl -s -o /dev/null -w "%{http_code}" -X POST "$URL" -H "Content-Type: application/json" -d '[]' # expect 2xx

try { writer.prepareCommit(); } catch (RuntimeException e) { Throwable root = ExceptionUtils.getRootCause(e); log.error("HTTP sink flush failed: {}", root.getMessage(), root); throw e; }

When it happens

Trigger: Using the HTTP sink in array mode (array_mode = true) and reaching checkpoint/commit while the target HTTP endpoint returns an error or the request fails (connection refused, timeout, 4xx/5xx).

Common situations: Endpoint temporarily down or rate-limiting at commit time, misconfigured URL or auth headers, or payload rejected by the receiving service.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

• Failed to flush data during prepareCommit()
• COMMIT_FAILED
• Single file mode is not supported when checkpoint is enabled
• Failed to flush data in prepareCommit
• FLUSH_DATA_FAILED

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/af53fe0e50a9d7c1. Report an issue: GitHub.