{"record":{"id":"dfaba046c0563972","repo":"prestodb/presto","slug":"hive-bad-data-dfaba0","errorCode":"HIVE_BAD_DATA","errorMessage":"Corrupted RC file: %s","messagePattern":"Corrupted RC file: (.+?)","errorType":"error_code","errorClass":"PrestoException","httpStatus":null,"severity":"error","filePath":"presto-hive/src/main/java/com/facebook/presto/hive/rcfile/RcFilePageSource.java","lineNumber":160,"sourceCode":"            Block[] blocks = new Block[hiveColumnIndexes.length];\n            for (int fieldId = 0; fieldId < blocks.length; fieldId++) {\n                if (constantBlocks[fieldId] != null) {\n                    blocks[fieldId] = new RunLengthEncodedBlock(constantBlocks[fieldId], currentPageSize);\n                }\n                else {\n                    blocks[fieldId] = createBlock(currentPageSize, fieldId);\n                }\n            }\n\n            return new Page(currentPageSize, blocks);\n        }\n        catch (PrestoException e) {\n            closeWithSuppression(e);\n            throw e;\n        }\n        catch (RcFileCorruptionException e) {\n            closeWithSuppression(e);\n            throw new PrestoException(HIVE_BAD_DATA, format(\"Corrupted RC file: %s\", rcFileReader.getId()), e);\n        }\n        catch (IOException | RuntimeException e) {\n            closeWithSuppression(e);\n            throw new PrestoException(HIVE_CURSOR_ERROR, format(\"Failed to read RC file: %s\", rcFileReader.getId()), e);\n        }\n    }\n\n    @Override\n    public void close()\n            throws IOException\n    {\n        // some hive input formats are broken and bad things can happen if you close them multiple times\n        if (closed) {\n            return;\n        }\n        closed = true;\n\n        rcFileReader.close();","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-hive/src/main/java/com/facebook/presto/hive/rcfile/RcFilePageSource.java#L142-L178","documentation":"Presto throws HIVE_BAD_DATA with 'Corrupted RC file: <id>' when the RC file reader signals an RcFileCorruptionException while advancing to the next page. This means the file's on-disk RCFile structure (header, metadata, or compressed row data) does not decode according to the RCFile spec, so reading cannot continue. The exception is raised from getNextPage during scan; the page source is closed with the corruption attached as the cause.","triggerScenarios":"rcFileReader.readBlock or page-advance logic throws RcFileCorruptionException inside getNextPage — typically when decoding a column block whose checksum/compression stream or record boundary is invalid mid-file.","commonSituations":"Files written by a buggy or incompatible RCFile writer, truncated/corrupt files after failed HDFS writes or manual copy, data moved between clusters with different compression codecs, or non-RCFile data stored with an .rcfile extension being read by the RCFile reader.","solutions":["Verify the file is intact: run 'hdfs fsck <path> -files -blocks' and re-copy or restore the corrupted block(s) from backup or upstream source.","Confirm the file really is RCFile: check the magic header bytes ('R\" + \"I\" + \"P\" + \"E' / 'RCF'); if it is actually ORC/Text/SequenceFile, fix the table's file format or storage format metadata.","Re-write the affected file(s) by re-running the producing job, or drop the corrupted file from the partition and re-ingest.","If the corruption is tolerated, use a skipped/failed-rows handling approach or exclude the bad file via partition pruning while investigating.","Upgrade/check Presto and Hive writer versions for known RCFile reader/writer incompatibilities."],"exampleFix":"// before: table declared with wrong format, reader hits corrupt stream\nCREATE TABLE t (...) WITH (format = 'RCFILE');\n// after: detect real format and set it correctly (e.g. data is ORC)\nCREATE TABLE t (...) WITH (format = 'ORC');","handlingStrategy":"validation","validationCode":"// Before scanning, sanity-check the file is present, non-empty, and structurally sound\nFileSystem fs = path.getFileSystem(conf);\nFileStatus st = fs.getFileStatus(path);\nif (st.getLen() == 0) throw new PrestoException(HIVE_BAD_DATA, \"RCFile is empty: \" + path);\ntry (FSDataInputStream in = fs.open(path)) {\n    byte[] magic = new byte[3];\n    if (in.readFully(magic) && !Arrays.equals(magic, new byte[]{'R','E','Q'} /* 'R\"+\"I\"+\"P\"+\"E' header check per spec */)) {\n        throw new PrestoException(HIVE_BAD_DATA, \"Not an RCFile: \" + path);\n    }\n}\nhdfs fsck <path> -files -blocks  // verify block health before querying","typeGuard":"// Java has no runtime type guard; narrow the cause explicitly:\npublic static boolean isRcFileCorruption(Throwable t) {\n    Throwable c = t;\n    while (c != null) {\n        if (c instanceof RcFileCorruptionException) return true;\n        c = c.getCause();\n    }\n    return false;\n}","tryCatchPattern":"try {\n    page = pageSource.getNextPage();\n} catch (PrestoException e) {\n    if (e.getErrorCode() == HIVE_BAD_DATA.toErrorCode() && isRcFileCorruption(e)) {\n        log.warn(\"Skipping corrupted RCFile: %s\", e.getCause());\n        // route to dead-letter / skip this split\n    } else {\n        throw e;\n    }\n}","preventionTips":["Run 'hdfs fsck' on table locations after bulk ingestion before exposing partitions.","Have writers emit to temp paths and atomically rename completed files.","Verify table storage format matches actual file magic bytes when migrating formats.","Keep writer and reader (Presto/Hive) versions compatible; test after upgrades.","Enable checksum verification on HDFS reads; do not disable it for performance."],"tags":["hive","rcfile","corrupt-data","hdfs"],"backgroundTag":"corrupted-data-file","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}