{"record":{"id":"201059119806da43","repo":"prestodb/presto","slug":"hive-bad-data","errorCode":"HIVE_BAD_DATA","errorMessage":"Line too long in text file: ","messagePattern":"Line too long in text file: ","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/GenericHiveRecordCursor.java","lineNumber":253,"sourceCode":"    {\n        try {\n            if (closed || !recordReader.next(key, value)) {\n                close();\n                return false;\n            }\n\n            // reset loaded flags\n            Arrays.fill(loaded, false);\n\n            // decode value\n            rowData = deserializer.deserialize(value);\n\n            return true;\n        }\n        catch (IOException | SerDeException | RuntimeException e) {\n            closeWithSuppression(this, e);\n            if (e instanceof TextLineLengthLimitExceededException) {\n                throw new PrestoException(HIVE_BAD_DATA, \"Line too long in text file: \" + path, e);\n            }\n            throw new PrestoException(HIVE_CURSOR_ERROR, e);\n        }\n    }\n\n    @Override\n    public boolean getBoolean(int fieldId)\n    {\n        checkState(!closed, \"Cursor is closed\");\n\n        validateType(fieldId, boolean.class);\n        if (!loaded[fieldId]) {\n            parseBooleanColumn(fieldId);\n        }\n        return booleans[fieldId];\n    }\n\n    private void parseBooleanColumn(int column)","sourceCodeStart":235,"sourceCodeEnd":271,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/GenericHiveRecordCursor.java#L235-L271","documentation":"While reading a Hive text-file table, GenericHiveRecordCursor.advanceNextPosition catches TextLineLengthLimitExceededException from the record reader and rethrows it as HIVE_BAD_DATA 'Line too long in text file: <path>'. The underlying TextInputFormat enforces a maximum line length (default 100MB-ish buffer limit); lines longer than this indicate corrupted or non-text data.","triggerScenarios":"Scanning a text/CSV Hive table whose file contains a single line exceeding the record reader's line-length limit — usually due to missing newlines, corrupted files, or binary data written as text.","commonSituations":"A CSV without proper line breaks (unterminated quoted field spanning everything), a log file that never flushes newlines, accidentally loading a binary/compressed-as-text file into a text table.","solutions":["Fix or regenerate the source file so lines are within the limit and properly newline-terminated","Inspect the file (e.g. awk to find lines longer than the limit) and split or repair it","Load the data as a correctly compressed/typed format (ORC/Parquet) instead of raw text","If the limit is genuinely too small, use a record reader/format with a larger max line length configuration"],"exampleFix":"// before: one giant line, no newline\nwriter.write(\"a,b,c\"); // ... never writes '\\n'\n// after\nwriter.write(\"a,b,c\\n\");","handlingStrategy":"try-catch","validationCode":"// pre-flight: detect overlong lines before reading via Hive\nlong maxLen = Files.lines(Paths.get(path)).mapToLong(String::length).max().orElse(0);\nif (maxLen > MAX_SUPPORTED_LINE_LENGTH) {\n    throw new IllegalStateException(\"File has lines exceeding limit: \" + path);\n}","typeGuard":null,"tryCatchPattern":"try {\n    cursor.advanceNextPosition();\n} catch (PrestoException e) {\n    if (\"HIVE_BAD_DATA\".equals(e.getErrorCode().getName()) && e.getMessage().startsWith(\"Line too long in text file:\")) {\n        // quarantine/skip the file, report path to the data pipeline owner\n    }\n    throw e;\n}","preventionTips":["Ensure writers always emit newline terminators","Validate text files (max line length) before loading into Hive tables","Store large blobs in a binary/object format, not text tables","Use ORC/Parquet for data without natural line boundaries"],"tags":["hive","text-file","data-corruption","record-cursor"],"backgroundTag":"line-too-long","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}