{"record":{"id":"31318283aaa7292f","repo":"prestodb/presto","slug":"corrupted-parquet-file-extra-d-values-to-be-cons","errorCode":null,"errorMessage":"Corrupted Parquet file: extra %d values to be consumed when scanning current batch","messagePattern":"Corrupted Parquet file: extra (.+?) values to be consumed when scanning current batch","errorType":"exception","errorClass":"ParquetDecodingException","httpStatus":null,"severity":"error","filePath":"presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/BooleanFlatBatchReader.java","lineNumber":222,"sourceCode":"        int remainingInBatch = nextBatchSize;\n        int startOffset = 0;\n        while (remainingInBatch > 0) {\n            if (remainingCountInPage == 0) {\n                if (!readNextPage()) {\n                    break;\n                }\n            }\n\n            int chunkSize = Math.min(remainingCountInPage, remainingInBatch);\n\n            valuesDecoder.readNext(values, startOffset, chunkSize);\n            startOffset += chunkSize;\n            remainingInBatch -= chunkSize;\n            remainingCountInPage -= chunkSize;\n        }\n\n        if (remainingInBatch != 0) {\n            throw new ParquetDecodingException(format(\"Corrupted Parquet file: extra %d values to be consumed when scanning current batch\", remainingInBatch));\n        }\n\n        Block block = new ByteArrayBlock(nextBatchSize, Optional.empty(), values);\n        return new ColumnChunk(block, new int[0], new int[0]);\n    }\n\n    private void seek()\n            throws IOException\n    {\n        if (readOffset == 0) {\n            return;\n        }\n\n        int remainingInBatch = readOffset;\n        int startOffset = 0;\n        while (remainingInBatch > 0) {\n            if (remainingCountInPage == 0) {\n                if (!readNextPage()) {","sourceCodeStart":204,"sourceCodeEnd":240,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/BooleanFlatBatchReader.java#L204-L240","documentation":"Thrown by BooleanFlatBatchReader.readWithoutNull when the loop over pages ends with remainingInBatch != 0, i.e. the page data did not supply all nextBatchSize values for a required (no-null) column. The library treats this as a corruption of the Parquet file because a required column chunk must contain exactly one value per row.","triggerScenarios":"Calling readNext on a required boolean column whose column chunk's pages collectively contain fewer values than the batch size computed from row-group/row counts — detected at the end of readWithoutNull's scanning loop.","commonSituations":"Files truncated mid-column-chunk; writers that wrote a wrong valueCount in the page header; hand-edited or re-chunked Parquet files; data copied from object storage with partial writes.","solutions":["Validate the file with parquet-tools / re-read the source table and regenerate the corrupted file","Check the writer that produced the file for page valueCount bugs (compare valueCount in page header to decoded values)","Upgrade Presto/parquet reader version — newer decoders surface clearer corruption messages","Restore the file from a known-good copy or checkpoint if storage truncation is suspected"],"exampleFix":"// before\nBlock b = reader.readNext().getBlock(); // throws on corrupt chunk\n// after\ntry {\n    Block b = reader.readNext().getBlock();\n} catch (ParquetDecodingException e) {\n    failoverToBackupFile();\n}","handlingStrategy":"try-catch","validationCode":"// ensure chunk is complete for required column\nif (chunkRowsRead + batchSize > columnChunkMetaData.getValueCount()) {\n    throw new IllegalStateException(\"batch overruns required column chunk\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    Block b = reader.readNext().getBlock();\n} catch (ParquetDecodingException e) {\n    failoverToBackupCopy();\n}","preventionTips":["Run parquet-tools dump on files before production use","Detect partial multipart uploads (ETag/size checks) before reading","Audit custom writers for page valueCount correctness","Pin a known-good Presto reader version"],"tags":["parquet","decoding","corrupt-file"],"backgroundTag":"parquet-decode-batch-short-read","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}