prestodb/presto · error · RcFileCorruptionException

Validation failed

Error message

Validation failed

What it means

RcFileReader.validateFile reads the file header/footer and column statistics during reader construction. Any IOException encountered while reading (truncated file, checksum failure, network/DFS read error) is wrapped in RcFileCorruptionException with the generic message 'Validation failed', signaling the RCFile is not readable as-is.

Source

Thrown at presto-rcfile/src/main/java/com/facebook/presto/rcfile/RcFileReader.java:540

        }
        try (RcFileReader rcFileReader = new RcFileReader(
                input,
                encoding,
                readTypes.build(),
                codecFactory,
                0,
                input.getSize(),
                new DataSize(8, Unit.MEGABYTE),
                Optional.of(writeValidation))) {
            while (rcFileReader.advance() >= 0) {
                // ignored
            }
        }
        catch (RcFileCorruptionException e) {
            throw e;
        }
        catch (IOException e) {
            throw new RcFileCorruptionException(e, "Validation failed");
        }
    }

    private static class Column
    {
        private final ColumnEncoding encoding;
        private final RcFileDecompressor decompressor;

        private BasicSliceInput lengthsInput;
        private Slice dataBuffer;
        private int uncompressedDataSize;

        private byte[] decompressedBuffer = new byte[0];

        private boolean compressed;

        private int currentPosition;

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify the file is complete and uncorrupted (re-copy or re-download the source file)
2. Check that the file is genuinely an RCFile with the expected header/magic bytes
3. Re-run the Hive job that produced the file if the source data was truncated
4. Catch RcFileCorruptionException and surface a user-facing 'corrupt file' error, possibly skipping the bad file

// before open: verify file exists and size is plausible
FileStatus st = fs.getFileStatus(path);
if (st.getLen() < RCFILE_MIN_HEADER_SIZE) { throw new IOException("File too small to be RCFile: " + path); }

try (RcFileReader r = new RcFileReader(...)) { ... }
catch (RcFileCorruptionException e) {
    log.error("RCFile failed validation: %s", path, e);
    throw new DataCorruptionException(path, e);
}

When it happens

Trigger: Opening an RCFile via RcFileReader when the underlying stream throws IOException during validation — truncated/corrupted file, wrong file (non-RCFile bytes), HDFS/S3 read failures mid-scan.

Common situations: Incomplete file uploads to object storage; HDFS block corruption; reading a file whose compression codec bytes disagree with the header; network interruptions when reading remote files.

Related errors

• HIVE_WRITER_CLOSE_ERROR
• HIVE_WRITER_OPEN_ERROR
• HIVE_CANNOT_OPEN_SPLIT
• Validation failed
• Read past end of RLE integer

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/faa3b4389189a760. Report an issue: GitHub.