prestodb/presto · error · ParquetDecodingException

Still remaining to be read in current batch.

Error message

Still remaining to be read in current batch.

What it means

ParquetDecodingException thrown by UuidFlatBatchReader.readWithNull() after decoding a batch when the loop consuming values leaves a nonzero remainingInBatch count. It means the decoder produced/consumed fewer values than the requested batch size, so the internal accounting between null/definition levels and value decoding is inconsistent — a sign of a corrupt page or a decoder bug.

Source

Thrown at presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/UuidFlatBatchReader.java:189

                int valueSourceIndex = startOffset + nonNullCount - 1;

                while (valueDestinationIndex >= startOffset) {
                    if (!isNull[valueDestinationIndex]) {
                        values[valueDestinationIndex * 2 + 1] = values[valueSourceIndex * 2 + 1];
                        values[valueDestinationIndex * 2] = values[valueSourceIndex * 2];
                        valueSourceIndex--;
                    }
                    valueDestinationIndex--;
                }
            }

            startOffset += chunkSize;
            remainingInBatch -= chunkSize;
            remainingCountInPage -= chunkSize;
        }

        if (remainingInBatch != 0) {
            throw new ParquetDecodingException("Still remaining to be read in current batch.");
        }

        if (totalNonNullCount == 0) {
            Block block = RunLengthEncodedBlock.create(field.getType(), null, nextBatchSize);
            return new ColumnChunk(block, new int[0], new int[0]);
        }

        boolean hasNoNull = totalNonNullCount == nextBatchSize;
        Block block = new Int128ArrayBlock(nextBatchSize, hasNoNull ? Optional.empty() : Optional.of(isNull), values);
        return new ColumnChunk(block, new int[0], new int[0]);
    }

    private ColumnChunk readWithoutNull()
            throws IOException
    {
        long[] values = new long[nextBatchSize * 2];
        int remainingInBatch = nextBatchSize;
        int startOffset = 0;

View on GitHub (pinned to 55bb57d202)

Solutions

1. Treat the file as corrupt: validate with parquet-tools and re-generate the file.
2. Check the page's definition-level count vs actual encoded values; inspect with a parquet metadata dumper.
3. Upgrade Presto/parquet reader version — decoder accounting bugs are fixed across releases.
4. If the file is produced in-house, fix the writer so non-null value counts match definition levels.

Example fix

// before
ColumnChunk chunk = reader.readNext(); // throws ParquetDecodingException: still remaining in batch
// after
ColumnChunk chunk;
try {
    chunk = reader.readNext();
} catch (ParquetDecodingException e) {
    // file/page corrupt: fail job with path context or route file to quarantine
    throw new RuntimeException("Corrupt UUID column page in " + parquetPath, e);
}

// pre-check: metadata value counts should match encoded values
long defined = rowGroup.getColumns(i).getMeta().getNumValues();
// mismatch vs page sizes implies corrupt file: validate with parquet-tools before reading

try {
    chunk = reader.readNext();
} catch (ParquetDecodingException e) {
    // quarantine the file, do not retry the same reader instance
    throw new RuntimeException("Corrupt UUID column page in " + path, e);
}

When it happens

Trigger: readWithNull() consumes definition levels and values in chunks inside its while loop; if after the loop remainingInBatch != 0 (e.g., a values decoder returns fewer non-null values than the definition levels indicate), this is thrown. Called from readNext() when the column has nulls.

Common situations: Parquet files written by writers that emit mismatched definition-level/value counts; truncated or corrupted page data; library bugs decoding UUID fixed_len_byte_array pages with nulls.

Related errors

• Corrupted Parquet file: extra %d values to be consumed when
• not a valid mode
• ICEBERG_BAD_DATA
• Unsupported Parquet encoding:
• Failed to decode.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/09f1e8e3ea240c62. Report an issue: GitHub.