prestodb/presto · error · PrestoException

HIVE_BAD_DATA

HIVE_BAD_DATA

Error message

Corrupted RC file: %s

What it means

Presto throws HIVE_BAD_DATA with 'Corrupted RC file: <id>' when the RC file reader signals an RcFileCorruptionException while advancing to the next page. This means the file's on-disk RCFile structure (header, metadata, or compressed row data) does not decode according to the RCFile spec, so reading cannot continue. The exception is raised from getNextPage during scan; the page source is closed with the corruption attached as the cause.

Source

Thrown at presto-hive/src/main/java/com/facebook/presto/hive/rcfile/RcFilePageSource.java:160

            Block[] blocks = new Block[hiveColumnIndexes.length];
            for (int fieldId = 0; fieldId < blocks.length; fieldId++) {
                if (constantBlocks[fieldId] != null) {
                    blocks[fieldId] = new RunLengthEncodedBlock(constantBlocks[fieldId], currentPageSize);
                }
                else {
                    blocks[fieldId] = createBlock(currentPageSize, fieldId);
                }
            }

            return new Page(currentPageSize, blocks);
        }
        catch (PrestoException e) {
            closeWithSuppression(e);
            throw e;
        }
        catch (RcFileCorruptionException e) {
            closeWithSuppression(e);
            throw new PrestoException(HIVE_BAD_DATA, format("Corrupted RC file: %s", rcFileReader.getId()), e);
        }
        catch (IOException | RuntimeException e) {
            closeWithSuppression(e);
            throw new PrestoException(HIVE_CURSOR_ERROR, format("Failed to read RC file: %s", rcFileReader.getId()), e);
        }
    }

    @Override
    public void close()
            throws IOException
    {
        // some hive input formats are broken and bad things can happen if you close them multiple times
        if (closed) {
            return;
        }
        closed = true;

        rcFileReader.close();

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify the file is intact: run 'hdfs fsck <path> -files -blocks' and re-copy or restore the corrupted block(s) from backup or upstream source.
2. Confirm the file really is RCFile: check the magic header bytes ('R" + "I" + "P" + "E' / 'RCF'); if it is actually ORC/Text/SequenceFile, fix the table's file format or storage format metadata.
3. Re-write the affected file(s) by re-running the producing job, or drop the corrupted file from the partition and re-ingest.
4. If the corruption is tolerated, use a skipped/failed-rows handling approach or exclude the bad file via partition pruning while investigating.
5. Upgrade/check Presto and Hive writer versions for known RCFile reader/writer incompatibilities.

Example fix

// before: table declared with wrong format, reader hits corrupt stream
CREATE TABLE t (...) WITH (format = 'RCFILE');
// after: detect real format and set it correctly (e.g. data is ORC)
CREATE TABLE t (...) WITH (format = 'ORC');

// Before scanning, sanity-check the file is present, non-empty, and structurally sound
FileSystem fs = path.getFileSystem(conf);
FileStatus st = fs.getFileStatus(path);
if (st.getLen() == 0) throw new PrestoException(HIVE_BAD_DATA, "RCFile is empty: " + path);
try (FSDataInputStream in = fs.open(path)) {
    byte[] magic = new byte[3];
    if (in.readFully(magic) && !Arrays.equals(magic, new byte[]{'R','E','Q'} /* 'R"+"I"+"P"+"E' header check per spec */)) {
        throw new PrestoException(HIVE_BAD_DATA, "Not an RCFile: " + path);
    }
}
hdfs fsck <path> -files -blocks  // verify block health before querying

// Java has no runtime type guard; narrow the cause explicitly:
public static boolean isRcFileCorruption(Throwable t) {
    Throwable c = t;
    while (c != null) {
        if (c instanceof RcFileCorruptionException) return true;
        c = c.getCause();
    }
    return false;
}

try {
    page = pageSource.getNextPage();
} catch (PrestoException e) {
    if (e.getErrorCode() == HIVE_BAD_DATA.toErrorCode() && isRcFileCorruption(e)) {
        log.warn("Skipping corrupted RCFile: %s", e.getCause());
        // route to dead-letter / skip this split
    } else {
        throw e;
    }
}

When it happens

Trigger: rcFileReader.readBlock or page-advance logic throws RcFileCorruptionException inside getNextPage — typically when decoding a column block whose checksum/compression stream or record boundary is invalid mid-file.

Common situations: Files written by a buggy or incompatible RCFile writer, truncated/corrupt files after failed HDFS writes or manual copy, data moved between clusters with different compression codecs, or non-RCFile data stored with an .rcfile extension being read by the RCFile reader.

Related errors

• HIVE_CURSOR_ERROR
• HIVE_BAD_DATA
• HIVE_CANNOT_OPEN_SPLIT
• HIVE_MISSING_DATA
• HIVE_FILESYSTEM_ERROR

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/dfaba046c0563972. Report an issue: GitHub.