apache/seatunnel · error · RuntimeException

Error queue overflow for stage [%s], plugin [%s]

Error message

Error queue overflow for stage [%s], plugin [%s]

What it means

Thrown when ErrorHandlerMode is FAIL (or unrecognized) and the bounded error queue cannot accept the new error row via its non-blocking offer. The row is rejected and the writer fails fast, signalling that the error queue for this stage/plugin is full and rows would be lost.

Source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/task/error/DefaultErrorSinkWriter.java:173

        try {
            switch (stageConfig.getQueueOverflowPolicy()) {
                case DROP:
                    pendingRows.incrementAndGet();
                    if (!queue.offer(errorRow)) {
                        pendingRows.decrementAndGet();
                        return false;
                    }
                    break;
                case BLOCK:
                    pendingRows.incrementAndGet();
                    enqueueWithBlockPolicy(ctx, errorRow);
                    break;
                case FAIL:
                default:
                    pendingRows.incrementAndGet();
                    if (!queue.offer(errorRow)) {
                        pendingRows.decrementAndGet();
                        throw new RuntimeException(
                                String.format(
                                        "Error queue overflow for stage [%s], plugin [%s]",
                                        ctx.getStage(), ctx.getPluginName()));
                    }
                    break;
            }
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            pendingRows.decrementAndGet();
            throw new RuntimeException("Interrupted while enqueuing error row for error sink", e);
        }
        return true;
    }

    private void enqueueWithBlockPolicy(RowErrorContext ctx, SeaTunnelRow errorRow)
            throws Exception {
        boolean enqueued = false;
        try {

View on GitHub (pinned to cf67b549a7)

Solutions

  1. Increase the error sink queue capacity configuration
  2. Switch ErrorHandlerMode to ROUTE with an appropriate blocking/overflow policy if backpressure is acceptable
  3. Investigate why the error sink worker is not draining the queue (downstream sink slowness, prior worker failure)
  4. Reduce the rate of bad rows by fixing upstream data quality or filters

Example fix

# before
error-sink {
  ErrorHandleMode = FAIL
  ErrorQueueCapacity = 10
}
# after
error-sink {
  ErrorHandleMode = ROUTE
  ErrorQueueCapacity = 10000
}
Defensive patterns

Strategy: validation

Validate before calling

long errorRate = estimateDirtyRowsPerSecond();
long capacity = errorQueueCapacity;
if (errorRate * flushIntervalMs / 1000 > capacity) {
    throw new IllegalArgumentException("ErrorQueueCapacity too small for expected dirty rate");
}

Try / catch

try {
    writer.write(row);
} catch (RuntimeException e) {
    if (e.getMessage() != null && e.getMessage().contains("Error queue overflow")) {
        LOG.warn("Error queue full, applying backpressure");
        // throttle or switch handle mode
    }
}

Prevention

When it happens

Trigger: ErrorHandlerMode.FAIL/default case in writeAndCheckAccepted: pendingRows incremented, queue.offer(errorRow) returns false because the queue capacity is exhausted (worker too slow or stopped draining), then pendingRows is decremented and this RuntimeException is thrown.

Common situations: Burst of bad rows exceeding the configured error queue capacity; error sink worker blocked writing downstream (slow/hung sink); misconfigured small queue size combined with high dirty-data rates.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/3a3057a30263c64a. Report an issue: GitHub.