{"record":{"id":"0169f56992bd8401","repo":"apache/seatunnel","slug":"too-many-row-level-errors-in-stage-s-plugin","errorCode":null,"errorMessage":"Too many row-level errors in stage [%s], plugin [%s]: %d records exceeded max_error_records=%d","messagePattern":"Too many row-level errors in stage \\[(.+?)\\], plugin \\[(.+?)\\]: (.+?) records exceeded max_error_records=(.+?)","errorType":"exception","errorClass":"RowErrorHandlingFatalException","httpStatus":null,"severity":"error","filePath":"seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/task/error/ErrorHandler.java","lineNumber":189,"sourceCode":"                        \"Error sink failed for stage [{}], plugin [{}], failing the job\",\n                        ctx.getStage(),\n                        ctx.getPluginName(),\n                        sinkEx);\n                throw new RowErrorHandlingFatalException(\n                        String.format(\n                                \"Error sink failed for stage [%s], plugin [%s]\",\n                                ctx.getStage(), ctx.getPluginName()),\n                        sinkEx);\n            }\n        }\n\n        maybeThrowOnThreshold(ctx, currentErrorCount);\n        return result;\n    }\n\n    private void maybeThrowOnThreshold(RowErrorContext ctx, long currentErrorCount) {\n        if (config.getMaxErrorRecords() > 0 && currentErrorCount > config.getMaxErrorRecords()) {\n            throw new RowErrorHandlingFatalException(\n                    String.format(\n                            \"Too many row-level errors in stage [%s], plugin [%s]: %d records exceeded max_error_records=%d\",\n                            stageName(ctx),\n                            pluginName(ctx),\n                            currentErrorCount,\n                            config.getMaxErrorRecords()));\n        }\n\n        maybeThrowOnRatioThreshold(ctx, currentErrorCount, counter.getTotalRecords());\n    }\n\n    private void maybeThrowOnRatioThreshold(\n            RowErrorContext ctx, long currentErrorCount, long total) {\n        // Only check ratio after min records threshold to ensure stability.\n        int minTotalForRatio =\n                config.getMaxErrorRatioMinRecords() > 0 ? config.getMaxErrorRatioMinRecords() : 1;\n        if (config.getMaxErrorRatio() > 0 && total >= minTotalForRatio) {\n            double ratio = (double) currentErrorCount / (double) total;","sourceCodeStart":171,"sourceCodeEnd":207,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/task/error/ErrorHandler.java#L171-L207","documentation":"RowErrorHandlingFatalException thrown by ErrorHandler.maybeThrowOnThreshold when the cumulative count of row-level errors in a stage exceeds the configured max_error_records limit. This is intentional job-level protection: once more bad rows than allowed have been observed, the job is failed instead of silently continuing.","triggerScenarios":"error-handler max_error_records is set to N>0 and error count reaches N+1 during row processing (checked from onError).","commonSituations":"Upstream data quality worse than expected; max_error_records set too low for normal dirty-data volume; a systematic mapping/parse bug producing errors on every row.","solutions":["Investigate the logged row errors to fix the root data or transform problem.","Raise max_error_records in the error-handler config if the current dirty-data volume is acceptable.","Add a transform or filter step to clean/repair known bad patterns before the failing plugin.","Set max_error_records=0 to disable the count threshold (not recommended for production)."],"exampleFix":"// before\nerror-handler { mode = IGNORE, max_error_records = 100 }\n// after\nerror-handler { mode = IGNORE, max_error_records = 10000 }","handlingStrategy":"validation","validationCode":"long maxErrorRecords = cfg.getLong(\"error-handler.max_error_records\");\nif (maxErrorRecords < 0) throw new IllegalArgumentException(\"max_error_records must be >= 0\");","typeGuard":null,"tryCatchPattern":"try { runJob(); } catch (RowErrorHandlingFatalException e) { if (e.getMessage().contains(\"max_error_records\")) { auditDirtyData(); } }","preventionTips":["Profile dirty-data rate on sample data before setting max_error_records","Set the limit well above expected dirty rows so only anomalies trip it","Log and alert on approaching-threshold counts rather than waiting for the fatal throw"],"tags":["row-level-error","threshold","config"],"backgroundTag":"threshold-exceeded","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}