{"record":{"id":"cc596c7ff58a8f8f","repo":"prestodb/presto","slug":"corrupted-parquet-file-extra-d-values-to-be-cons-cc596c","errorCode":null,"errorMessage":"Corrupted Parquet file: extra %d values to be consumed when scanning current batch","messagePattern":"Corrupted Parquet file: extra (.+?) values to be consumed when scanning current batch","errorType":"exception","errorClass":"ParquetDecodingException","httpStatus":null,"severity":"error","filePath":"presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/Int64FlatBatchReader.java","lineNumber":222,"sourceCode":"        int remainingInBatch = nextBatchSize;\n        int startOffset = 0;\n        while (remainingInBatch > 0) {\n            if (remainingCountInPage == 0) {\n                if (!readNextPage()) {\n                    break;\n                }\n            }\n\n            int chunkSize = Math.min(remainingCountInPage, remainingInBatch);\n\n            valuesDecoder.readNext(values, startOffset, chunkSize);\n            startOffset += chunkSize;\n            remainingInBatch -= chunkSize;\n            remainingCountInPage -= chunkSize;\n        }\n\n        if (remainingInBatch != 0) {\n            throw new ParquetDecodingException(format(\"Corrupted Parquet file: extra %d values to be consumed when scanning current batch\", remainingInBatch));\n        }\n\n        Block block = new LongArrayBlock(nextBatchSize, Optional.empty(), values);\n        return new ColumnChunk(block, new int[0], new int[0]);\n    }\n\n    private void seek()\n            throws IOException\n    {\n        if (readOffset == 0) {\n            return;\n        }\n\n        int remainingInBatch = readOffset;\n        int startOffset = 0;\n        while (remainingInBatch > 0) {\n            if (remainingCountInPage == 0) {\n                if (!readNextPage()) {","sourceCodeStart":204,"sourceCodeEnd":240,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/Int64FlatBatchReader.java#L204-L240","documentation":"Int64FlatBatchReader.readWithoutNull throws this when a required int64 column chunk's pages delivered fewer values than the requested batch size. Required columns must contain exactly one value per row, so a shortfall is reported as a corrupted Parquet file.","triggerScenarios":"readNext on a required int64 column whose pages end before nextBatchSize values are scanned — remainingInBatch != 0 after readWithoutNull's loop.","commonSituations":"Truncated or incompletely written files; page headers with wrong valueCount; files re-chunked or repaired incorrectly by third-party tools; partial S3 multipart uploads.","solutions":["Validate with parquet-tools; regenerate the corrupt file from source data","Audit the producing writer for page valueCount bugs","Restore a known-good copy of the file","Upgrade Presto reader version for clearer corruption errors"],"exampleFix":"// before\nBlock b = reader.readNext().getBlock();\n// after\ntry {\n    Block b = reader.readNext().getBlock();\n} catch (ParquetDecodingException e) {\n    switchToBackupCopy();\n}","handlingStrategy":"try-catch","validationCode":"if (!columnDescriptor.isOptional() && rowsRemainingInChunk < batchSize) {\n    throw new IllegalStateException(\"required int64 chunk under-filled\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    reader.readNext();\n} catch (ParquetDecodingException e) {\n    switchToBackupCopyAndLog(e);\n}","preventionTips":["Check for partial S3 multipart uploads (size/ETag)","Validate files with parquet-tools before querying","Audit third-party parquet repair/re-chunk tools","Enable end-to-end checksums on storage"],"tags":["parquet","decoding","corrupt-file"],"backgroundTag":"parquet-decode-batch-short-read","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}