prestodb/presto · error · ParquetDecodingException

Corrupted Parquet file: extra %d values to be consumed when

Error message

Corrupted Parquet file: extra %d values to be consumed when scanning current batch

What it means

Int64FlatBatchReader.readWithoutNull throws this when a required int64 column chunk's pages delivered fewer values than the requested batch size. Required columns must contain exactly one value per row, so a shortfall is reported as a corrupted Parquet file.

Source

Thrown at presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/Int64FlatBatchReader.java:222

        int remainingInBatch = nextBatchSize;
        int startOffset = 0;
        while (remainingInBatch > 0) {
            if (remainingCountInPage == 0) {
                if (!readNextPage()) {
                    break;
                }
            }

            int chunkSize = Math.min(remainingCountInPage, remainingInBatch);

            valuesDecoder.readNext(values, startOffset, chunkSize);
            startOffset += chunkSize;
            remainingInBatch -= chunkSize;
            remainingCountInPage -= chunkSize;
        }

        if (remainingInBatch != 0) {
            throw new ParquetDecodingException(format("Corrupted Parquet file: extra %d values to be consumed when scanning current batch", remainingInBatch));
        }

        Block block = new LongArrayBlock(nextBatchSize, Optional.empty(), values);
        return new ColumnChunk(block, new int[0], new int[0]);
    }

    private void seek()
            throws IOException
    {
        if (readOffset == 0) {
            return;
        }

        int remainingInBatch = readOffset;
        int startOffset = 0;
        while (remainingInBatch > 0) {
            if (remainingCountInPage == 0) {
                if (!readNextPage()) {

View on GitHub (pinned to 55bb57d202)

Solutions

Validate with parquet-tools; regenerate the corrupt file from source data
Audit the producing writer for page valueCount bugs
Restore a known-good copy of the file
Upgrade Presto reader version for clearer corruption errors

Example fix

// before
Block b = reader.readNext().getBlock();
// after
try {
    Block b = reader.readNext().getBlock();
} catch (ParquetDecodingException e) {
    switchToBackupCopy();
}

Defensive patterns

Strategy: try-catch

Validate before calling

if (!columnDescriptor.isOptional() && rowsRemainingInChunk < batchSize) {
    throw new IllegalStateException("required int64 chunk under-filled");
}

Try / catch

try {
    reader.readNext();
} catch (ParquetDecodingException e) {
    switchToBackupCopyAndLog(e);
}

Prevention

Check for partial S3 multipart uploads (size/ETag)
Validate files with parquet-tools before querying
Audit third-party parquet repair/re-chunk tools
Enable end-to-end checksums on storage

When it happens

Trigger: readNext on a required int64 column whose pages end before nextBatchSize values are scanned — remainingInBatch != 0 after readWithoutNull's loop.

Common situations: Truncated or incompletely written files; page headers with wrong valueCount; files re-chunked or repaired incorrectly by third-party tools; partial S3 multipart uploads.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/cc596c7ff58a8f8f. Report an issue: GitHub.