{"record":{"id":"6aa078c4673b969d","repo":"prestodb/presto","slug":"hive-bad-data-6aa078","errorCode":"HIVE_BAD_DATA","errorMessage":"Line too long in text file: %s","messagePattern":"Line too long in text file: (.+?)","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/HiveUtil.java","lineNumber":320,"sourceCode":"        try {\n            RecordReader<WritableComparable, Writable> recordReader = (RecordReader<WritableComparable, Writable>) inputFormat.getRecordReader(fileSplit, jobConf, Reporter.NULL);\n\n            int headerCount = getHeaderCount(schema);\n            //  Only skip header rows when the split is at the beginning of the file\n            if (start == 0 && headerCount > 0) {\n                Utilities.skipHeader(recordReader, headerCount, recordReader.createKey(), recordReader.createValue());\n            }\n\n            int footerCount = getFooterCount(schema);\n            if (footerCount > 0) {\n                recordReader = new FooterAwareRecordReader<>(recordReader, footerCount, jobConf);\n            }\n\n            return recordReader;\n        }\n        catch (IOException e) {\n            if (e instanceof TextLineLengthLimitExceededException) {\n                throw new PrestoException(HIVE_BAD_DATA, \"Line too long in text file: \" + path, e);\n            }\n\n            throw new PrestoException(HIVE_CANNOT_OPEN_SPLIT, format(\"Error opening Hive split %s (offset=%s, length=%s) using %s: %s\",\n                    path,\n                    start,\n                    length,\n                    getInputFormatName(schema),\n                    firstNonNull(e.getMessage(), e.getClass().getName())),\n                    e);\n        }\n    }\n\n    public static void setReadColumns(Configuration configuration, List<Integer> readHiveColumnIndexes)\n    {\n        configuration.set(READ_COLUMN_IDS_CONF_STR, Joiner.on(',').join(readHiveColumnIndexes));\n        configuration.setBoolean(READ_ALL_COLUMNS, false);\n    }\n","sourceCodeStart":302,"sourceCodeEnd":338,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/HiveUtil.java#L302-L338","documentation":"A text-format file contains a line longer than the maximum allowed by the LineRecordReader (default 10MB per line). HiveUtil.createRecordReader converts the TextLineLengthLimitExceededException from the underlying reader into HIVE_BAD_DATA with the file path, signaling corrupt or malformed source data rather than a connector bug.","triggerScenarios":"Reading a Hive text/textfile split whose single line exceeds the record reader's max line length — typically a file missing newline terminators, a corrupted text file, or binary data (e.g. gzip-less avoro/sequence bytes) written as text.","commonSituations":"Unterminated final line in a text file produced by a broken writer; log files with one giant line; accidentally registering non-text data as TEXTFILE format.","solutions":["Repair or re-write the source file with proper line breaks and correct format","Increase the per-line limit via jobconf if genuinely long lines are expected (mapreduce.input.linerecordreader.line.maxlength)","Change the table/storage format to one matching the actual data (e.g. sequence/parquet for binary data)"],"exampleFix":"-- before: binary data declared as TEXTFILE\nCREATE TABLE t WITH (format = 'TEXTFILE') ...\n-- after\nCREATE TABLE t WITH (format = 'PARQUET') ... -- match the real file format","handlingStrategy":"try-catch","validationCode":"long maxLen = maxLineLengthBytes(\"hdfs://path/file\"); // precompute longest line, compare to record reader limit\nif (maxLen > 10L * 1024 * 1024) throw new IllegalStateException(\"file has lines exceeding text record reader limit\");","typeGuard":null,"tryCatchPattern":"try { readSplit(split); } catch (PrestoException e) { if (e.getErrorCode().getCode() == StandardErrorCode.HIVE_BAD_DATA.getCode()) { log.error(\"Oversized line in {}\", path); /* quarantine file or use a format with no line limit */ } else throw e; }","preventionTips":["Validate text file line lengths and newline termination before loading into Hive tables","Store binary data in binary formats (parquet/orc/sequence) instead of TEXTFILE","Set an explicit larger line-maxlength jobconf when long lines are legitimate"],"tags":["hive","bad-data","text-file","corrupt-data"],"backgroundTag":"line-too-long-in-text-file","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}