apache/seatunnel · error · RowErrorHandlingFatalException

Row-level error ratio in stage

Error message

Row-level error ratio in stage [%s], plugin [%s] exceeded max_error_ratio=%.4f (current=%.4f, errors=%d, total=%d)

What it means

RowErrorHandlingFatalException thrown by ErrorHandler.maybeThrowOnRatioThreshold when the fraction of row-level errors (errors/total processed rows) exceeds max_error_ratio after at least max_error_ratio_min_records rows have been processed. It fails the job to stop processing when error rates indicate systemic data problems.

Solutions

1. Fix the upstream data or the transform causing the high error rate (see logged row errors).
2. Lower expectations vs reality: increase max_error_ratio to a value matching acceptable quality.
3. Increase max_error_ratio_min_records so early batches don't trigger on small-sample ratios.
4. Set max_error_ratio=0.0 to disable the ratio check if only the count threshold is wanted.

Example fix

// before
error-handler { max_error_ratio = 0.01, max_error_ratio_min_records = 10 }
// after
error-handler { max_error_ratio = 0.05, max_error_ratio_min_records = 1000 }

double ratio = cfg.getDouble("error-handler.max_error_ratio");
int minRecs = cfg.getInt("error-handler.max_error_ratio_min_records");
if (ratio < 0 || ratio > 1) throw new IllegalArgumentException("max_error_ratio in [0,1]");
if (ratio > 0 && minRecs <= 0) warn("set max_error_ratio_min_records to avoid small-sample triggers");

try { runJob(); } catch (RowErrorHandlingFatalException e) { if (e.getMessage().contains("max_error_ratio")) { auditErrorRateAndData(); } }

When it happens

Trigger: max_error_ratio > 0 configured; total processed rows >= min-records warmup; currentErrorCount/total > max_error_ratio — triggered either from incrementTotalRecords or at threshold check in onError.

Common situations: A source delivers corrupted or wrongly-typed rows in bulk; ratio threshold set tighter than real data quality; warmup min-records set so low that early bursts trip the ratio.

Related errors

• All fields in row are null, skipping
• DATA_DESERIALIZE_FAILED
• DATA_TYPE_CAST_FAILED
• Error sink failed for stage
• Field is null, setting to NULL in prepared statement

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/15ed40213f148271. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/task/error/ErrorHandler.java:209

                            "Too many row-level errors in stage [%s], plugin [%s]: %d records exceeded max_error_records=%d",
                            stageName(ctx),
                            pluginName(ctx),
                            currentErrorCount,
                            config.getMaxErrorRecords()));
        }

        maybeThrowOnRatioThreshold(ctx, currentErrorCount, counter.getTotalRecords());
    }

    private void maybeThrowOnRatioThreshold(
            RowErrorContext ctx, long currentErrorCount, long total) {
        // Only check ratio after min records threshold to ensure stability.
        int minTotalForRatio =
                config.getMaxErrorRatioMinRecords() > 0 ? config.getMaxErrorRatioMinRecords() : 1;
        if (config.getMaxErrorRatio() > 0 && total >= minTotalForRatio) {
            double ratio = (double) currentErrorCount / (double) total;
            if (ratio > config.getMaxErrorRatio()) {
                throw new RowErrorHandlingFatalException(
                        String.format(
                                "Row-level error ratio in stage [%s], plugin [%s] exceeded max_error_ratio=%.4f (current=%.4f, errors=%d, total=%d)",
                                stageName(ctx),
                                pluginName(ctx),
                                config.getMaxErrorRatio(),
                                ratio,
                                currentErrorCount,
                                total));
            }
        }
    }

    private String stageName(RowErrorContext ctx) {
        return ctx == null ? "UNKNOWN" : ctx.getStage();
    }

    private String pluginName(RowErrorContext ctx) {
        return ctx == null ? "UNKNOWN" : ctx.getPluginName();

View on GitHub (pinned to cf67b549a7)