{"record":{"id":"09f1e8e3ea240c62","repo":"prestodb/presto","slug":"still-remaining-to-be-read-in-current-batch-09f1e8","errorCode":null,"errorMessage":"Still remaining to be read in current batch.","messagePattern":"Still remaining to be read in current batch\\.","errorType":"exception","errorClass":"ParquetDecodingException","httpStatus":null,"severity":"error","filePath":"presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/UuidFlatBatchReader.java","lineNumber":189,"sourceCode":"                int valueSourceIndex = startOffset + nonNullCount - 1;\n\n                while (valueDestinationIndex >= startOffset) {\n                    if (!isNull[valueDestinationIndex]) {\n                        values[valueDestinationIndex * 2 + 1] = values[valueSourceIndex * 2 + 1];\n                        values[valueDestinationIndex * 2] = values[valueSourceIndex * 2];\n                        valueSourceIndex--;\n                    }\n                    valueDestinationIndex--;\n                }\n            }\n\n            startOffset += chunkSize;\n            remainingInBatch -= chunkSize;\n            remainingCountInPage -= chunkSize;\n        }\n\n        if (remainingInBatch != 0) {\n            throw new ParquetDecodingException(\"Still remaining to be read in current batch.\");\n        }\n\n        if (totalNonNullCount == 0) {\n            Block block = RunLengthEncodedBlock.create(field.getType(), null, nextBatchSize);\n            return new ColumnChunk(block, new int[0], new int[0]);\n        }\n\n        boolean hasNoNull = totalNonNullCount == nextBatchSize;\n        Block block = new Int128ArrayBlock(nextBatchSize, hasNoNull ? Optional.empty() : Optional.of(isNull), values);\n        return new ColumnChunk(block, new int[0], new int[0]);\n    }\n\n    private ColumnChunk readWithoutNull()\n            throws IOException\n    {\n        long[] values = new long[nextBatchSize * 2];\n        int remainingInBatch = nextBatchSize;\n        int startOffset = 0;","sourceCodeStart":171,"sourceCodeEnd":207,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/UuidFlatBatchReader.java#L171-L207","documentation":"ParquetDecodingException thrown by UuidFlatBatchReader.readWithNull() after decoding a batch when the loop consuming values leaves a nonzero remainingInBatch count. It means the decoder produced/consumed fewer values than the requested batch size, so the internal accounting between null/definition levels and value decoding is inconsistent — a sign of a corrupt page or a decoder bug.","triggerScenarios":"readWithNull() consumes definition levels and values in chunks inside its while loop; if after the loop remainingInBatch != 0 (e.g., a values decoder returns fewer non-null values than the definition levels indicate), this is thrown. Called from readNext() when the column has nulls.","commonSituations":"Parquet files written by writers that emit mismatched definition-level/value counts; truncated or corrupted page data; library bugs decoding UUID fixed_len_byte_array pages with nulls.","solutions":["Treat the file as corrupt: validate with parquet-tools and re-generate the file.","Check the page's definition-level count vs actual encoded values; inspect with a parquet metadata dumper.","Upgrade Presto/parquet reader version — decoder accounting bugs are fixed across releases.","If the file is produced in-house, fix the writer so non-null value counts match definition levels."],"exampleFix":"// before\nColumnChunk chunk = reader.readNext(); // throws ParquetDecodingException: still remaining in batch\n// after\nColumnChunk chunk;\ntry {\n    chunk = reader.readNext();\n} catch (ParquetDecodingException e) {\n    // file/page corrupt: fail job with path context or route file to quarantine\n    throw new RuntimeException(\"Corrupt UUID column page in \" + parquetPath, e);\n}","handlingStrategy":"validation","validationCode":"// pre-check: metadata value counts should match encoded values\nlong defined = rowGroup.getColumns(i).getMeta().getNumValues();\n// mismatch vs page sizes implies corrupt file: validate with parquet-tools before reading","typeGuard":null,"tryCatchPattern":"try {\n    chunk = reader.readNext();\n} catch (ParquetDecodingException e) {\n    // quarantine the file, do not retry the same reader instance\n    throw new RuntimeException(\"Corrupt UUID column page in \" + path, e);\n}","preventionTips":["Validate files with parquet-tools before ingestion","Regenerate files from trustworthy writers","Keep the reader library up to date (decoder accounting fixes)","Match reader and writer Parquet format versions"],"tags":["parquet","decoding","corruption","uuid"],"backgroundTag":"parquet-decode-count-mismatch","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}