{"record":{"id":"39502e886f4b71f5","repo":"prestodb/presto","slug":"corrupted-parquet-file-extra-d-values-to-be-cons-39502e","errorCode":null,"errorMessage":"Corrupted Parquet file: extra %d values to be consumed when scanning current batch","messagePattern":"Corrupted Parquet file: extra (.+?) values to be consumed when scanning current batch","errorType":"exception","errorClass":"ParquetDecodingException","httpStatus":null,"severity":"error","filePath":"presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/UuidFlatBatchReader.java","lineNumber":224,"sourceCode":"        int remainingInBatch = nextBatchSize;\n        int startOffset = 0;\n        while (remainingInBatch > 0) {\n            if (remainingCountInPage == 0) {\n                if (!readNextPage()) {\n                    break;\n                }\n            }\n\n            int chunkSize = min(remainingCountInPage, remainingInBatch);\n\n            valuesDecoder.readNext(values, startOffset, chunkSize);\n            startOffset += chunkSize;\n            remainingInBatch -= chunkSize;\n            remainingCountInPage -= chunkSize;\n        }\n\n        if (remainingInBatch != 0) {\n            throw new ParquetDecodingException(format(\"Corrupted Parquet file: extra %d values to be consumed when scanning current batch\", remainingInBatch));\n        }\n\n        Block block = new Int128ArrayBlock(nextBatchSize, Optional.empty(), values);\n        return new ColumnChunk(block, new int[0], new int[0]);\n    }\n\n    private void seek()\n            throws IOException\n    {\n        if (readOffset == 0) {\n            return;\n        }\n\n        int remainingInBatch = readOffset;\n        int startOffset = 0;\n        while (remainingInBatch > 0) {\n            if (remainingCountInPage == 0) {\n                if (!readNextPage()) {","sourceCodeStart":206,"sourceCodeEnd":242,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/UuidFlatBatchReader.java#L206-L242","documentation":"ParquetDecodingException thrown by UuidFlatBatchReader.readWithoutNull() when, after consuming all chunks for a batch, remainingInBatch is nonzero — the page contains more encoded values than the declared batch size. The library treats this as file corruption ('extra %d values to be consumed') because a no-null column chunk must consume exactly nextBatchSize values.","triggerScenarios":"readWithoutNull() (called from readNext() when the column is declared non-nullable) reads chunks in a loop; if the page still has extra values after filling nextBatchSize, this is thrown with the leftover count.","commonSituations":"Files written by writers whose page value counts exceed the chunk's declared count; corrupted or hand-edited Parquet files; mismatched metadata when a file is partially overwritten.","solutions":["Validate and regenerate the Parquet file; verify page counts with parquet-tools dump.","Check that the file wasn't truncated/partially rewritten by the producer.","Upgrade the Presto reader — earlier versions had bugs tolerating writer quirks.","If you control the writer, ensure page value counts match row-group/chunk metadata."],"exampleFix":"// before\nColumnChunk chunk = reader.readNext(); // throws: extra N values to be consumed\n// after\ntry {\n    chunk = reader.readNext();\n} catch (ParquetDecodingException e) {\n    throw new RuntimeException(\"Corrupt non-null UUID page (extra values) in \" + parquetPath, e);\n}","handlingStrategy":"validation","validationCode":"// verify declared chunk value count equals page values before decoding\nlong chunkValues = chunkMeta.getNumValues();\nif (chunkValues != expectedBatchTotal) throw new IllegalStateException(\"Chunk value count mismatch: \" + path);","typeGuard":null,"tryCatchPattern":"try {\n    chunk = reader.readNext();\n} catch (ParquetDecodingException e) {\n    // treat file as corrupt; quarantine with path context\n    throw new RuntimeException(\"Corrupt non-null UUID page in \" + path, e);\n}","preventionTips":["Validate page value counts against chunk metadata before reading","Avoid hand-editing or partially rewriting Parquet files","Keep reader version current","Regenerate files that fail parquet-tools dump"],"tags":["parquet","decoding","corruption","uuid"],"backgroundTag":"parquet-decode-count-mismatch","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}