{"record":{"id":"756a55b16daa6e9d","repo":"prestodb/presto","slug":"still-remaining-to-be-read-in-current-batch-756a55","errorCode":null,"errorMessage":"Still remaining to be read in current batch.","messagePattern":"Still remaining to be read in current batch\\.","errorType":"exception","errorClass":"ParquetDecodingException","httpStatus":null,"severity":"error","filePath":"presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/Int64TimeAndTimestampMicrosFlatBatchReader.java","lineNumber":187,"sourceCode":"                int valueDestinationIndex = startOffset + chunkSize - 1;\n                int valueSourceIndex = startOffset + nonNullCount - 1;\n\n                while (valueDestinationIndex >= startOffset) {\n                    if (!isNull[valueDestinationIndex]) {\n                        values[valueDestinationIndex] = values[valueSourceIndex];\n                        valueSourceIndex--;\n                    }\n                    valueDestinationIndex--;\n                }\n            }\n\n            startOffset += chunkSize;\n            remainingInBatch -= chunkSize;\n            remainingCountInPage -= chunkSize;\n        }\n\n        if (remainingInBatch != 0) {\n            throw new ParquetDecodingException(\"Still remaining to be read in current batch.\");\n        }\n\n        if (totalNonNullCount == 0) {\n            Block block = RunLengthEncodedBlock.create(field.getType(), null, nextBatchSize);\n            return new ColumnChunk(block, new int[0], new int[0]);\n        }\n\n        boolean hasNoNull = totalNonNullCount == nextBatchSize;\n        Block block = new LongArrayBlock(nextBatchSize, hasNoNull ? Optional.empty() : Optional.of(isNull), values);\n        return new ColumnChunk(block, new int[0], new int[0]);\n    }\n\n    private ColumnChunk readWithoutNull()\n            throws IOException\n    {\n        long[] values = new long[nextBatchSize];\n        int remainingInBatch = nextBatchSize;\n        int startOffset = 0;","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/Int64TimeAndTimestampMicrosFlatBatchReader.java#L169-L205","documentation":"Int64TimeAndTimestampMicrosFlatBatchReader.readWithNull throws this when the batch of nextBatchSize micros time/timestamp values could not be fully filled after all pages of the column chunk were consumed. Page value counts (including null entries via definition levels) do not match the data actually present.","triggerScenarios":"readNext on an optional TIME_MICROS/TIMESTAMP_MICROS column where pages run out (readNextPage() returns null) before nextBatchSize values are decoded, leaving remainingInBatch > 0.","commonSituations":"Truncated Parquet files; writers emitting incorrect page valueCount for temporal columns; corrupted RLE definition-level streams; partial writes to object storage.","solutions":["Validate and regenerate the file; confirm the column chunk is complete with parquet-tools","Fix the writer's page metadata if files are produced in-house","Re-download or restore the file; check for truncated S3/HDFS writes","Upgrade Presto for better corruption detection and messages"],"exampleFix":"// before\nreader.read(batchSize); // throws when pages exhausted early\n// after\nint rows = (int) Math.min(batchSize, chunkRowsRemaining);\nreader.read(rows);","handlingStrategy":"try-catch","validationCode":"long avail = columnChunkMetaData.getValueCount() - rowsRead;\nif (batchSize > avail) { batchSize = (int) avail; }","typeGuard":null,"tryCatchPattern":"try {\n    reader.readNext();\n} catch (ParquetDecodingException e) {\n    handleCorruptTemporalColumn(e); // rescan or failover\n}","preventionTips":["Validate temporal column chunks after writing (parquet-tools)","Guard against truncated object writes on S3/HDFS","Fix writer page valueCount accounting for micros time/timestamp","Upgrade Presto reader for stricter corruption checks"],"tags":["parquet","decoding","corrupt-file","timestamp"],"backgroundTag":"parquet-decode-batch-short-read","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}