apache/flink · error · IOException

could not read page {} in col {}

Error message

could not read page {} in col {}

What it means

IOException from readPageV1 in AbstractColumnReader: while extracting page bytes and initializing the repetition-level reader, run-length decoder, and value decoder (prepareNewPage), an IOException was thrown - malformed or truncated page payload for the column. The page and column descriptor are embedded in the message and the root cause is chained.

Source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/reader/AbstractColumnReader.java:216

    private void readPageV1(DataPageV1 page) throws IOException {
        this.pageValueCount = page.getValueCount();
        ValuesReader rlReader = page.getRlEncoding().getValuesReader(descriptor, REPETITION_LEVEL);

        // Initialize the decoders.
        if (page.getDlEncoding() != Encoding.RLE && descriptor.getMaxDefinitionLevel() != 0) {
            throw new UnsupportedOperationException(
                    "Unsupported encoding: " + page.getDlEncoding());
        }
        int bitWidth = BytesUtils.getWidthFromMaxInt(descriptor.getMaxDefinitionLevel());
        this.runLenDecoder = new RunLengthDecoder(bitWidth);
        try {
            BytesInput bytes = page.getBytes();
            ByteBufferInputStream in = bytes.toInputStream();
            rlReader.initFromPage(pageValueCount, in);
            this.runLenDecoder.initFromStream(pageValueCount, in);
            prepareNewPage(page.getValueEncoding(), in);
        } catch (IOException e) {
            throw new IOException("could not read page " + page + " in col " + descriptor, e);
        }
    }

    private void readPageV2(DataPageV2 page) throws IOException {
        this.pageValueCount = page.getValueCount();

        int bitWidth = BytesUtils.getWidthFromMaxInt(descriptor.getMaxDefinitionLevel());
        // do not read the length from the stream. v2 pages handle dividing the page bytes.
        this.runLenDecoder = new RunLengthDecoder(bitWidth, false);
        this.runLenDecoder.initFromStream(
                this.pageValueCount, page.getDefinitionLevels().toInputStream());
        try {
            prepareNewPage(page.getDataEncoding(), page.getData().toInputStream());
        } catch (IOException e) {
            throw new IOException("could not read page " + page + " in col " + descriptor, e);
        }
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Check the chained cause: EOF/truncation points to incomplete files, codec errors to compression mismatches
2. Re-copy or regenerate the file; verify checksums/Etags before reading
3. Ensure the compression codec used by the writer is on Flink's classpath and versions match
4. Isolate the bad split from the error message (col descriptor) and reprocess only that data

catch (IOException e) { if (e.getMessage() != null && e.getMessage().startsWith("could not read page")) { if (isTransientStorageError(e.getCause())) { reDownloadOrRetrySplit(); } else { quarantineCorruptFile(path); } } else throw e; }

When it happens

Trigger: DataPageV1.initFromPage/prepareNewPage failing while parsing page bytes for the given column descriptor - truncated page, corrupt compression stream, or values that violate the declared encoding.

Common situations: Truncated files from interrupted writes or partial uploads; bit rot / transfer corruption on object storage; compression codec mismatches between writer and reader (e.g. unsupported codec variant).

Related errors

• expecting more rows but reached last block. Read {} out of {
• could not decode the dictionary for {}
• totalValueCount == 0
• Corrupted Parquet schema
• Unknown ColumnIO, %s

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/b2475e0505f932bc. Report an issue: GitHub.