{"record":{"id":"771c3caf5e9f39ab","repo":"prestodb/presto","slug":"still-remaining-to-be-read-in-current-batch-771c3c","errorCode":null,"errorMessage":"Still remaining to be read in current batch.","messagePattern":"Still remaining to be read in current batch\\.","errorType":"exception","errorClass":"ParquetDecodingException","httpStatus":null,"severity":"error","filePath":"presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/Int64FlatBatchReader.java","lineNumber":187,"sourceCode":"                int valueDestinationIndex = startOffset + chunkSize - 1;\n                int valueSourceIndex = startOffset + nonNullCount - 1;\n\n                while (valueDestinationIndex >= startOffset) {\n                    if (!isNull[valueDestinationIndex]) {\n                        values[valueDestinationIndex] = values[valueSourceIndex];\n                        valueSourceIndex--;\n                    }\n                    valueDestinationIndex--;\n                }\n            }\n\n            startOffset += chunkSize;\n            remainingInBatch -= chunkSize;\n            remainingCountInPage -= chunkSize;\n        }\n\n        if (remainingInBatch != 0) {\n            throw new ParquetDecodingException(\"Still remaining to be read in current batch.\");\n        }\n\n        if (totalNonNullCount == 0) {\n            Block block = RunLengthEncodedBlock.create(field.getType(), null, nextBatchSize);\n            return new ColumnChunk(block, new int[0], new int[0]);\n        }\n\n        boolean hasNoNull = totalNonNullCount == nextBatchSize;\n        Block block = new LongArrayBlock(nextBatchSize, hasNoNull ? Optional.empty() : Optional.of(isNull), values);\n        return new ColumnChunk(block, new int[0], new int[0]);\n    }\n\n    private ColumnChunk readWithoutNull()\n            throws IOException\n    {\n        long[] values = new long[nextBatchSize];\n        int remainingInBatch = nextBatchSize;\n        int startOffset = 0;","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/Int64FlatBatchReader.java#L169-L205","documentation":"Int64FlatBatchReader.readWithNull throws this when the batch of nextBatchSize int64 values could not be fully read after consuming all pages of the column chunk. Page metadata promised more values (counting nulls via definition levels) than the pages actually contained.","triggerScenarios":"readNext on an optional int64 column where total page valueCounts < nextBatchSize; readNextPage() exhausts all pages while remainingInBatch > 0.","commonSituations":"Truncated column chunks; writer bugs producing wrong page valueCount; corrupted definition-level (RLE) data ending early.","solutions":["Validate and regenerate the file (parquet-tools or full table rewrite)","Fix the writer's page value-count accounting if the file is internally produced","Upgrade Presto to a version with stricter/earlier corruption detection","Re-download the file; look for truncated object-storage writes"],"exampleFix":"// before\nreader.read(batchSize);\n// after\nint safe = (int) Math.min(batchSize, remainingRowsInChunk);\nreader.read(safe);","handlingStrategy":"try-catch","validationCode":"long avail = columnChunkMetaData.getValueCount() - rowsRead;\nif (batchSize > avail) { batchSize = (int) avail; }","typeGuard":null,"tryCatchPattern":"try {\n    reader.readNext();\n} catch (ParquetDecodingException e) {\n    failoverOrRescanFromGoodCopy();\n}","preventionTips":["Validate Parquet metadata against data after writes","Detect truncated column chunks via footer checks","Pin compatible writer/reader versions","Keep backup copies of critical datasets"],"tags":["parquet","decoding","corrupt-file"],"backgroundTag":"parquet-decode-batch-short-read","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}