apache/seatunnel · error · RowErrorHandlingFatalException

Too many row-level errors in stage

Error message

Too many row-level errors in stage [%s], plugin [%s]: %d records exceeded max_error_records=%d

What it means

RowErrorHandlingFatalException thrown by ErrorHandler.maybeThrowOnThreshold when the cumulative count of row-level errors in a stage exceeds the configured max_error_records limit. This is intentional job-level protection: once more bad rows than allowed have been observed, the job is failed instead of silently continuing.

Solutions

1. Investigate the logged row errors to fix the root data or transform problem.
2. Raise max_error_records in the error-handler config if the current dirty-data volume is acceptable.
3. Add a transform or filter step to clean/repair known bad patterns before the failing plugin.
4. Set max_error_records=0 to disable the count threshold (not recommended for production).

Example fix

// before
error-handler { mode = IGNORE, max_error_records = 100 }
// after
error-handler { mode = IGNORE, max_error_records = 10000 }

long maxErrorRecords = cfg.getLong("error-handler.max_error_records");
if (maxErrorRecords < 0) throw new IllegalArgumentException("max_error_records must be >= 0");

try { runJob(); } catch (RowErrorHandlingFatalException e) { if (e.getMessage().contains("max_error_records")) { auditDirtyData(); } }

When it happens

Trigger: error-handler max_error_records is set to N>0 and error count reaches N+1 during row processing (checked from onError).

Common situations: Upstream data quality worse than expected; max_error_records set too low for normal dirty-data volume; a systematic mapping/parse bug producing errors on every row.

Related errors

• error handler max_error_records must be non-negative, but…
• accessId and accesskey must be provided when sts_token is…
• Agent config is not a readable file
• agent.id must be non-empty after resolution.
• At least one sink plugin must be configured.

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/0169f56992bd8401. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/task/error/ErrorHandler.java:189

                        "Error sink failed for stage [{}], plugin [{}], failing the job",
                        ctx.getStage(),
                        ctx.getPluginName(),
                        sinkEx);
                throw new RowErrorHandlingFatalException(
                        String.format(
                                "Error sink failed for stage [%s], plugin [%s]",
                                ctx.getStage(), ctx.getPluginName()),
                        sinkEx);
            }
        }

        maybeThrowOnThreshold(ctx, currentErrorCount);
        return result;
    }

    private void maybeThrowOnThreshold(RowErrorContext ctx, long currentErrorCount) {
        if (config.getMaxErrorRecords() > 0 && currentErrorCount > config.getMaxErrorRecords()) {
            throw new RowErrorHandlingFatalException(
                    String.format(
                            "Too many row-level errors in stage [%s], plugin [%s]: %d records exceeded max_error_records=%d",
                            stageName(ctx),
                            pluginName(ctx),
                            currentErrorCount,
                            config.getMaxErrorRecords()));
        }

        maybeThrowOnRatioThreshold(ctx, currentErrorCount, counter.getTotalRecords());
    }

    private void maybeThrowOnRatioThreshold(
            RowErrorContext ctx, long currentErrorCount, long total) {
        // Only check ratio after min records threshold to ensure stability.
        int minTotalForRatio =
                config.getMaxErrorRatioMinRecords() > 0 ? config.getMaxErrorRatioMinRecords() : 1;
        if (config.getMaxErrorRatio() > 0 && total >= minTotalForRatio) {
            double ratio = (double) currentErrorCount / (double) total;

View on GitHub (pinned to cf67b549a7)