prestodb/presto · error · ParquetDecodingException

Corrupted Parquet file: extra %d values to be consumed when

Error message

Corrupted Parquet file: extra %d values to be consumed when scanning current batch

What it means

Thrown by BooleanFlatBatchReader.readWithoutNull when the loop over pages ends with remainingInBatch != 0, i.e. the page data did not supply all nextBatchSize values for a required (no-null) column. The library treats this as a corruption of the Parquet file because a required column chunk must contain exactly one value per row.

Source

Thrown at presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/BooleanFlatBatchReader.java:222

        int remainingInBatch = nextBatchSize;
        int startOffset = 0;
        while (remainingInBatch > 0) {
            if (remainingCountInPage == 0) {
                if (!readNextPage()) {
                    break;
                }
            }

            int chunkSize = Math.min(remainingCountInPage, remainingInBatch);

            valuesDecoder.readNext(values, startOffset, chunkSize);
            startOffset += chunkSize;
            remainingInBatch -= chunkSize;
            remainingCountInPage -= chunkSize;
        }

        if (remainingInBatch != 0) {
            throw new ParquetDecodingException(format("Corrupted Parquet file: extra %d values to be consumed when scanning current batch", remainingInBatch));
        }

        Block block = new ByteArrayBlock(nextBatchSize, Optional.empty(), values);
        return new ColumnChunk(block, new int[0], new int[0]);
    }

    private void seek()
            throws IOException
    {
        if (readOffset == 0) {
            return;
        }

        int remainingInBatch = readOffset;
        int startOffset = 0;
        while (remainingInBatch > 0) {
            if (remainingCountInPage == 0) {
                if (!readNextPage()) {

View on GitHub (pinned to 55bb57d202)

Solutions

1. Validate the file with parquet-tools / re-read the source table and regenerate the corrupted file
2. Check the writer that produced the file for page valueCount bugs (compare valueCount in page header to decoded values)
3. Upgrade Presto/parquet reader version — newer decoders surface clearer corruption messages
4. Restore the file from a known-good copy or checkpoint if storage truncation is suspected

Example fix

// before
Block b = reader.readNext().getBlock(); // throws on corrupt chunk
// after
try {
    Block b = reader.readNext().getBlock();
} catch (ParquetDecodingException e) {
    failoverToBackupFile();
}

// ensure chunk is complete for required column
if (chunkRowsRead + batchSize > columnChunkMetaData.getValueCount()) {
    throw new IllegalStateException("batch overruns required column chunk");
}

try {
    Block b = reader.readNext().getBlock();
} catch (ParquetDecodingException e) {
    failoverToBackupCopy();
}

When it happens

Trigger: Calling readNext on a required boolean column whose column chunk's pages collectively contain fewer values than the batch size computed from row-group/row counts — detected at the end of readWithoutNull's scanning loop.

Common situations: Files truncated mid-column-chunk; writers that wrote a wrong valueCount in the page header; hand-edited or re-chunked Parquet files; data copied from object storage with partial writes.

Related errors

• We didn't read correct number of definitionLevels
• Still remaining to be read in current batch.
• Still remaining to be read in current batch.
• Corrupted Parquet file: extra %d values to be consumed when
• Still remaining to be read in current batch.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/31318283aaa7292f. Report an issue: GitHub.