prestodb/presto · error · ParquetDecodingException

Corrupted Parquet file: extra %d values to be consumed when

Error message

Corrupted Parquet file: extra %d values to be consumed when scanning current batch

What it means

Thrown by TimestampFlatBatchReader.readWithoutNull when a required TIMESTAMP column chunk lacks enough raw INT64 values to fill the requested batch. The loop consumed every page of the chunk while values were still owed (remainingInBatch > 0), meaning the declared value count does not match the encoded data. The library labels this 'Corrupted Parquet file' with the outstanding value count and aborts the scan.

Source

Thrown at presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/TimestampFlatBatchReader.java:222

        int remainingInBatch = nextBatchSize;
        int startOffset = 0;
        while (remainingInBatch > 0) {
            if (remainingCountInPage == 0) {
                if (!readNextPage()) {
                    break;
                }
            }

            int chunkSize = Math.min(remainingCountInPage, remainingInBatch);

            valuesDecoder.readNext(values, startOffset, chunkSize, timezone);
            startOffset += chunkSize;
            remainingInBatch -= chunkSize;
            remainingCountInPage -= chunkSize;
        }

        if (remainingInBatch != 0) {
            throw new ParquetDecodingException(format("Corrupted Parquet file: extra %d values to be consumed when scanning current batch", remainingInBatch));
        }

        Block block = new LongArrayBlock(nextBatchSize, Optional.empty(), values);
        return new ColumnChunk(block, new int[0], new int[0]);
    }

    private void seek()
            throws IOException
    {
        if (readOffset == 0) {
            return;
        }

        int remainingInBatch = readOffset;
        int startOffset = 0;
        while (remainingInBatch > 0) {
            if (remainingCountInPage == 0) {
                if (!readNextPage()) {

View on GitHub (pinned to 55bb57d202)

Solutions

Re-transfer the file and verify checksum/size against the producer.
Regenerate the file with a completed writer job; check job logs for errors.
Validate value counts per page/row group using parquet-tools or a metadata reader.
Fix/upgrade the external writer if page headers are incorrect.
Exclude the corrupt row group/file from the scan and continue with the rest.

Example fix

// before
ColumnChunk chunk = reader.readNext(timezone); // extra N values owed

// after
long valueCount = colMeta.getValueCount();
if (colChunkLength < valueCount * 8L) { quarantine(path); return; }
ColumnChunk chunk = reader.readNext(timezone);

Defensive patterns

Strategy: validation

Validate before calling

long valueCount = colMeta.getValueCount();
if (colChunkLength < valueCount * 8L) { // required int64 timestamp
    throw new IOException("Required timestamp chunk byte-count mismatch: " + path);
}

Try / catch

try {
    return reader.readNext(timezone);
} catch (ParquetDecodingException e) {
    quarantineFile(path, e);
    throw new PrestoException(PARQUET_BAD_DATA, "Corrupt timestamp column chunk", e);
}

Prevention

Stage files and atomically publish so truncated writes are never queryable.
Verify checksums/sizes on upload completion (multipart-complete verification).
Run post-write metadata validation comparing declared vs actual value counts.
Standardize on one writer library/version to avoid header inconsistencies.

When it happens

Trigger: readNext(timezone) on a required TIMESTAMP column when readNextPage() exhausts the column chunk while remainingInBatch > 0 — fewer encoded values exist than the batch/metadata demands.

Common situations: Truncated or partially uploaded files, interrupted Hive/Spark writes, object-store corruption or bit rot, incompatible writer versions emitting wrong page valueCounts, files manually concatenated or edited after writing.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/7bac91f86761cc95. Report an issue: GitHub.