{"record":{"id":"15ed40213f148271","repo":"apache/seatunnel","slug":"row-level-error-ratio-in-stage-s-plugin-s-e","errorCode":null,"errorMessage":"Row-level error ratio in stage [%s], plugin [%s] exceeded max_error_ratio=%.4f (current=%.4f, errors=%d, total=%d)","messagePattern":"Row-level error ratio in stage \\[(.+?)\\], plugin \\[(.+?)\\] exceeded max_error_ratio=%\\.4f \\(current=%\\.4f, errors=(.+?), total=(.+?)\\)","errorType":"exception","errorClass":"RowErrorHandlingFatalException","httpStatus":null,"severity":"error","filePath":"seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/task/error/ErrorHandler.java","lineNumber":209,"sourceCode":"                            \"Too many row-level errors in stage [%s], plugin [%s]: %d records exceeded max_error_records=%d\",\n                            stageName(ctx),\n                            pluginName(ctx),\n                            currentErrorCount,\n                            config.getMaxErrorRecords()));\n        }\n\n        maybeThrowOnRatioThreshold(ctx, currentErrorCount, counter.getTotalRecords());\n    }\n\n    private void maybeThrowOnRatioThreshold(\n            RowErrorContext ctx, long currentErrorCount, long total) {\n        // Only check ratio after min records threshold to ensure stability.\n        int minTotalForRatio =\n                config.getMaxErrorRatioMinRecords() > 0 ? config.getMaxErrorRatioMinRecords() : 1;\n        if (config.getMaxErrorRatio() > 0 && total >= minTotalForRatio) {\n            double ratio = (double) currentErrorCount / (double) total;\n            if (ratio > config.getMaxErrorRatio()) {\n                throw new RowErrorHandlingFatalException(\n                        String.format(\n                                \"Row-level error ratio in stage [%s], plugin [%s] exceeded max_error_ratio=%.4f (current=%.4f, errors=%d, total=%d)\",\n                                stageName(ctx),\n                                pluginName(ctx),\n                                config.getMaxErrorRatio(),\n                                ratio,\n                                currentErrorCount,\n                                total));\n            }\n        }\n    }\n\n    private String stageName(RowErrorContext ctx) {\n        return ctx == null ? \"UNKNOWN\" : ctx.getStage();\n    }\n\n    private String pluginName(RowErrorContext ctx) {\n        return ctx == null ? \"UNKNOWN\" : ctx.getPluginName();","sourceCodeStart":191,"sourceCodeEnd":227,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/task/error/ErrorHandler.java#L191-L227","documentation":"RowErrorHandlingFatalException thrown by ErrorHandler.maybeThrowOnRatioThreshold when the fraction of row-level errors (errors/total processed rows) exceeds max_error_ratio after at least max_error_ratio_min_records rows have been processed. It fails the job to stop processing when error rates indicate systemic data problems.","triggerScenarios":"max_error_ratio > 0 configured; total processed rows >= min-records warmup; currentErrorCount/total > max_error_ratio — triggered either from incrementTotalRecords or at threshold check in onError.","commonSituations":"A source delivers corrupted or wrongly-typed rows in bulk; ratio threshold set tighter than real data quality; warmup min-records set so low that early bursts trip the ratio.","solutions":["Fix the upstream data or the transform causing the high error rate (see logged row errors).","Lower expectations vs reality: increase max_error_ratio to a value matching acceptable quality.","Increase max_error_ratio_min_records so early batches don't trigger on small-sample ratios.","Set max_error_ratio=0.0 to disable the ratio check if only the count threshold is wanted."],"exampleFix":"// before\nerror-handler { max_error_ratio = 0.01, max_error_ratio_min_records = 10 }\n// after\nerror-handler { max_error_ratio = 0.05, max_error_ratio_min_records = 1000 }","handlingStrategy":"validation","validationCode":"double ratio = cfg.getDouble(\"error-handler.max_error_ratio\");\nint minRecs = cfg.getInt(\"error-handler.max_error_ratio_min_records\");\nif (ratio < 0 || ratio > 1) throw new IllegalArgumentException(\"max_error_ratio in [0,1]\");\nif (ratio > 0 && minRecs <= 0) warn(\"set max_error_ratio_min_records to avoid small-sample triggers\");","typeGuard":null,"tryCatchPattern":"try { runJob(); } catch (RowErrorHandlingFatalException e) { if (e.getMessage().contains(\"max_error_ratio\")) { auditErrorRateAndData(); } }","preventionTips":["Measure real error ratios on historical data before choosing max_error_ratio","Always pair the ratio with a meaningful max_error_ratio_min_records warmup","Alert on error-ratio trends so the job doesn't become the first detector"],"tags":["row-level-error","ratio-threshold","data-quality"],"backgroundTag":"threshold-exceeded","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}