prestodb/presto · error · ParquetDecodingException

Still remaining to be read in current batch.

Error message

Still remaining to be read in current batch.

What it means

Int64TimeAndTimestampMicrosFlatBatchReader.readWithNull throws this when the batch of nextBatchSize micros time/timestamp values could not be fully filled after all pages of the column chunk were consumed. Page value counts (including null entries via definition levels) do not match the data actually present.

Source

Thrown at presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/Int64TimeAndTimestampMicrosFlatBatchReader.java:187

                int valueDestinationIndex = startOffset + chunkSize - 1;
                int valueSourceIndex = startOffset + nonNullCount - 1;

                while (valueDestinationIndex >= startOffset) {
                    if (!isNull[valueDestinationIndex]) {
                        values[valueDestinationIndex] = values[valueSourceIndex];
                        valueSourceIndex--;
                    }
                    valueDestinationIndex--;
                }
            }

            startOffset += chunkSize;
            remainingInBatch -= chunkSize;
            remainingCountInPage -= chunkSize;
        }

        if (remainingInBatch != 0) {
            throw new ParquetDecodingException("Still remaining to be read in current batch.");
        }

        if (totalNonNullCount == 0) {
            Block block = RunLengthEncodedBlock.create(field.getType(), null, nextBatchSize);
            return new ColumnChunk(block, new int[0], new int[0]);
        }

        boolean hasNoNull = totalNonNullCount == nextBatchSize;
        Block block = new LongArrayBlock(nextBatchSize, hasNoNull ? Optional.empty() : Optional.of(isNull), values);
        return new ColumnChunk(block, new int[0], new int[0]);
    }

    private ColumnChunk readWithoutNull()
            throws IOException
    {
        long[] values = new long[nextBatchSize];
        int remainingInBatch = nextBatchSize;
        int startOffset = 0;

View on GitHub (pinned to 55bb57d202)

Solutions

1. Validate and regenerate the file; confirm the column chunk is complete with parquet-tools
2. Fix the writer's page metadata if files are produced in-house
3. Re-download or restore the file; check for truncated S3/HDFS writes
4. Upgrade Presto for better corruption detection and messages

Example fix

// before
reader.read(batchSize); // throws when pages exhausted early
// after
int rows = (int) Math.min(batchSize, chunkRowsRemaining);
reader.read(rows);

long avail = columnChunkMetaData.getValueCount() - rowsRead;
if (batchSize > avail) { batchSize = (int) avail; }

try {
    reader.readNext();
} catch (ParquetDecodingException e) {
    handleCorruptTemporalColumn(e); // rescan or failover
}

When it happens

Trigger: readNext on an optional TIME_MICROS/TIMESTAMP_MICROS column where pages run out (readNextPage() returns null) before nextBatchSize values are decoded, leaving remainingInBatch > 0.

Common situations: Truncated Parquet files; writers emitting incorrect page valueCount for temporal columns; corrupted RLE definition-level streams; partial writes to object storage.

Related errors

• Still remaining to be read in current batch.
• Corrupted Parquet file: extra %d values to be consumed when
• We didn't read correct number of definitionLevels
• Still remaining to be read in current batch.
• Corrupted Parquet file: extra %d values to be consumed when

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/756a55b16daa6e9d. Report an issue: GitHub.