{"record":{"id":"fa7feb82a6f3b20b","repo":"apache/flink","slug":"the-record-length-exceeded-the-maximum-record-leng","errorCode":null,"errorMessage":"The record length exceeded the maximum record length (${lineLengthLimit}).","messagePattern":"The record length exceeded the maximum record length \\((.+?)\\)\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/api/common/io/DelimitedInputFormat.java","lineNumber":663,"sourceCode":"                        this.wrapBuffer = nb;\n                    }\n                    if (count >= 0) {\n                        System.arraycopy(\n                                this.readBuffer, 0, this.wrapBuffer, countInWrapBuffer, count);\n                    }\n                    setResult(this.wrapBuffer, 0, countInWrapBuffer + count);\n                    return true;\n                } else {\n                    setResult(this.readBuffer, startPos, count);\n                    return true;\n                }\n            } else {\n                // we reached the end of the readBuffer\n                count = this.limit - startPos;\n\n                // check against the maximum record length\n                if (((long) countInWrapBuffer) + count > this.lineLengthLimit) {\n                    throw new IOException(\n                            \"The record length exceeded the maximum record length (\"\n                                    + this.lineLengthLimit\n                                    + \").\");\n                }\n\n                // Compute number of bytes to move to wrapBuffer\n                // Chars of partially read delimiter must remain in the readBuffer. We might need to\n                // go back.\n                int bytesToMove = count - delimPos;\n                // ensure wrapBuffer is large enough\n                if (this.wrapBuffer.length - countInWrapBuffer < bytesToMove) {\n                    // reallocate\n                    byte[] tmp =\n                            new byte\n                                    [Math.max(\n                                            this.wrapBuffer.length * 2,\n                                            countInWrapBuffer + bytesToMove)];\n                    System.arraycopy(this.wrapBuffer, 0, tmp, 0, countInWrapBuffer);","sourceCodeStart":645,"sourceCodeEnd":681,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/api/common/io/DelimitedInputFormat.java#L645-L681","documentation":"While reading, DelimitedInputFormat accumulates bytes for the current record into a wrap buffer when a record spans the end of the read buffer. If the accumulated record length exceeds lineLengthLimit, it throws IOException — this is a safety valve to stop runaway reads caused by a wrong/missing delimiter (the whole file would otherwise be treated as one record).","triggerScenarios":"Reading a file where the configured delimiter never appears within lineLengthLimit bytes (wrong delimiter, binary file read as text, or a genuinely oversized record).","commonSituations":"Wrong delimiter for the file format (e.g. '\\n' set but file uses '\\r\\n' or a custom separator); reading a binary/compressed-without-decompression file as delimited text; lineLengthLimit left at default on files with legitimately long records.","solutions":["Verify and correct the delimiter to match the actual file format (use a hex viewer / od to confirm).","Increase lineLengthLimit via setLineLengthLimit to accommodate the largest legitimate record.","Decompress the file first (or let the format decompress via the appropriate InflaterInputStreamFactory / file extension).","If the file is binary, use a dedicated InputFormat rather than the delimited text format."],"exampleFix":"// before: wrong delimiter -> whole file read as one record -> throws\nformat.setDelimiter(\";\");\nformat.setLineLengthLimit(1024);\n// file uses newlines\n\n// after\nformat.setDelimiter(\"\\n\");\nformat.setLineLengthLimit(10 * 1024 * 1024);","handlingStrategy":"try-catch","validationCode":"// Validate the delimiter actually appears in a sample of the file before reading.\n// (defensive; the real guard is catching IOException at runtime)\nformat.setLineLengthLimit(largestExpectedRecordBytes);","typeGuard":null,"tryCatchPattern":"try {\n    while (!format.reachedEnd()) {\n        OT rec = format.nextRecord(null);\n        // process\n    }\n} catch (IOException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"maximum record length\")) {\n        // delimiter likely wrong; raise a clearer error or switch delimiter\n        throw new RuntimeException(\"Record exceeded lineLengthLimit; verify delimiter and limit\", e);\n    }\n    throw e;\n}","preventionTips":["Confirm the delimiter with a hex dump of a sample before configuring the format.","Set lineLengthLimit above the largest legitimate record size.","Decompress inputs before delimited reading; do not read binary as text."],"tags":["input-format","delimited","record-length","delimiter","ioexception","data-quality"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}