{"record":{"id":"69dce3e579067f16","repo":"apache/flink","slug":"expecting-more-rows-but-reached-last-block-read-69dce3","errorCode":null,"errorMessage":"expecting more rows but reached last block. Read {} out of {}","messagePattern":"expecting more rows but reached last block\\. Read (.+?) out of (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/ParquetColumnarRowSplitReader.java","lineNumber":320,"sourceCode":"        if (rowsReturned == totalCountLoadedSoFar) {\n            readNextRowGroup();\n        }\n\n        int num = (int) Math.min(batchSize, totalCountLoadedSoFar - rowsReturned);\n        for (int i = 0; i < columnReaders.length; ++i) {\n            //noinspection unchecked\n            columnReaders[i].readToVector(num, writableVectors[i]);\n        }\n        rowsReturned += num;\n        columnarBatch.setNumRows(num);\n        rowsInBatch = num;\n        return true;\n    }\n\n    private void readNextRowGroup() throws IOException {\n        PageReadStore pages = reader.readNextRowGroup();\n        if (pages == null) {\n            throw new IOException(\n                    \"expecting more rows but reached last block. Read \"\n                            + rowsReturned\n                            + \" out of \"\n                            + totalRowCount);\n        }\n        List<Type> types = requestedSchema.getFields();\n        columnReaders = new ColumnReader[types.size()];\n        for (int i = 0; i < types.size(); ++i) {\n            columnReaders[i] =\n                    createColumnReader(\n                            utcTimestamp,\n                            selectedTypes[i],\n                            types.get(i),\n                            requestedSchema.getColumns(),\n                            pages,\n                            fieldList.get(i),\n                            0);\n        }","sourceCodeStart":302,"sourceCodeEnd":338,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/ParquetColumnarRowSplitReader.java#L302-L338","documentation":"IOException from readNextRowGroup() in ParquetColumnarRowSplitReader: the underlying ParquetFileReader.readNextRowGroup() returned null (no more row groups) while the reader still expects rows because rowsReturned has not reached totalRowCount (the row count advertised in the file footer). This is a data/metadata inconsistency: the footer promised more rows than the row groups contain.","triggerScenarios":"readNextRowGroup() called when reader.readNextRowGroup() returns null, i.e. all row groups consumed while rowsReturned < totalRowCount. Typically triggered by truncated or corrupted files, or writers that wrote inconsistent footer row-count metadata.","commonSituations":"Files corrupted or truncated by interrupted writes, network copies, or faulty object storage; files produced by non-standard writers with wrong row-group row counts; combining seekToRow offsets derived from stale metadata.","solutions":["Verify the file integrity: parquet-tools meta / cat on the exact split file, and re-copy or regenerate it if corrupted","Re-run the job that produced the file so the footer is written atomically and completely","Check object-storage consistency (S3 read-after-write, eventual consistency) if reading files immediately after upload","If the file is valid, check that the split's row-count offset math (seekToRow usage) is not skipping into the middle of row groups"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"ParquetMetadata md = ParquetFileReader.readFooter(conf, path);\nlong footerRows = md.getBlocks().stream().mapToLong(BlockMetaData::getRowCount).sum();\nif (footerRows != md.getBlocks().get(md.getBlocks().size()-1).getRowCount() * 0L + footerRows) { /* sanity placeholder */ }\n// real check: ensure sum of block row counts equals advertised total before reading","typeGuard":null,"tryCatchPattern":"catch (IOException e) { if (e.getMessage().contains(\"reached last block\")) { /* quarantine corrupt split, regenerate source file, then retry */ } else throw e; }","preventionTips":["Write Parquet atomically (temp file + rename) so footers are never half-written","Validate files with parquet-tools cat --rows after production jobs"],"tags":["parquet","corruption","row-group","io","flink"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}