apache/seatunnel · error · RuntimeException

Failed to flush data during prepareCommit()

Error message

Failed to flush data during prepareCommit()

What it means

HugeGraphSinkWriter.flushes its buffered records inside prepareCommit() so data is durable before the checkpoint completes. If buffer.flush() throws, the writer logs the failure and rethrows it wrapped in a RuntimeException so the checkpoint fails and the pipeline can retry from the last successful checkpoint. This prevents silent data loss between checkpoints.

Source

Thrown at seatunnel-connectors-v2/connector-hugegraph/src/main/java/org/apache/seatunnel/connectors/seatunnel/hugegraph/sink/HugeGraphSinkWriter.java:829

        if (pendingUpdateBefore != null) {
            throw new HugeGraphConnectorException(
                    HugeGraphConnectorErrorCode.GRAPH_OPERATION_FAILED,
                    "Checkpoint requested between UPDATE_BEFORE and UPDATE_AFTER: "
                            + "UPDATE_BEFORE received but its paired UPDATE_AFTER has "
                            + "not yet arrived. The SeaTunnel SinkWriter interface does "
                            + "not support persisting in-flight mutation state across "
                            + "checkpoints. UPDATE_BEFORE and UPDATE_AFTER must arrive "
                            + "within the same checkpoint interval. "
                            + "Mitigations: (1) increase the checkpoint interval, "
                            + "(2) use INSERT-only mode if the source does not emit "
                            + "changelog events.");
        }

        try {
            buffer.flush();
        } catch (Exception e) {
            LOG.error("Failed to flush data during prepareCommit, failing checkpoint.", e);
            throw new RuntimeException("Failed to flush data during prepareCommit()", e);
        }
        return Optional.empty();
    }

    @Override
    public void close() throws IOException {
        Exception failure = null;
        try {
            if (buffer != null) {
                buffer.close();
            }
        } catch (Exception e) {
            failure = e;
        } finally {
            try {
                if (client != null) {
                    client.close();
                }

View on GitHub (pinned to cf67b549a7)

Solutions

  1. Check the LOG.error stack trace for the root cause (connection refused, timeout, server-side validation) and address that underlying issue first.
  2. Verify HugeGraph server availability and connectivity (host, port, credentials) from the SeaTunnel worker node.
  3. Increase request timeouts / reduce the write buffer size so each flush batch is smaller and less likely to time out.
  4. Validate that buffered records conform to the HugeGraph schema (label/property definitions) to avoid server-side rejections.
  5. Rely on checkpoint retry: after fixing the cause, the failed checkpoint is retried and the buffer is re-flushed from replayed records.

Example fix

// before
buffer.flush();
// after
try {
    buffer.flush();
} catch (Exception e) {
    LOG.error("Failed to flush data during prepareCommit, failing checkpoint.", e);
    throw new RuntimeException("Failed to flush data during prepareCommit()", e);
}
Defensive patterns

Strategy: try-catch

Validate before calling

// before submit: verify HugeGraph reachable
// curl -s http://HOST:PORT/apis/version || exit 1

Try / catch

try {
  writer.prepareCommit(state);
} catch (RuntimeException e) {
  LOG.error("checkpoint flush failed", e);
  // let checkpoint fail; engine will retry from last checkpoint
}

Prevention

When it happens

Trigger: A SeaTunnel checkpoint fires, prepareCommit() is invoked, and the HugeGraph server rejects or times out during the Gremlin/gRPC write of buffered records, throwing an exception caught around buffer.flush().

Common situations: HugeGraph server temporarily down or restarting; network interruption between SeaTunnel worker and HugeGraph; huge batch causing request timeouts; invalid data (bad property values or schema mismatch) that the server rejects mid-flush; authentication/token expiration.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/6796c09605e0b0d9. Report an issue: GitHub.