apache/flink · error · IOException

totalValueCount == 0

Error message

totalValueCount == 0

What it means

IOException from the AbstractColumnReader constructor: pageReader.getTotalValueCount() returned 0 for the column chunk in the current row group. A column reader is only constructed when values are expected (the row group has rows), so a zero value count means empty/miscomputed chunk statistics - usually corrupt or pathological metadata rather than a legitimate all-null column (all-null columns still have definition-level values counted).

Source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/reader/AbstractColumnReader.java:128

        DictionaryPage dictionaryPage = pageReader.readDictionaryPage();
        if (dictionaryPage != null) {
            try {
                this.dictionary =
                        dictionaryPage.getEncoding().initDictionary(descriptor, dictionaryPage);
                this.isCurrentPageDictionaryEncoded = true;
            } catch (IOException e) {
                throw new IOException("could not decode the dictionary for " + descriptor, e);
            }
        } else {
            this.dictionary = null;
            this.isCurrentPageDictionaryEncoded = false;
        }
        /*
         * Total number of values in this column (in this row group).
         */
        long totalValueCount = pageReader.getTotalValueCount();
        if (totalValueCount == 0) {
            throw new IOException("totalValueCount == 0");
        }
    }

    protected void checkTypeName(PrimitiveType.PrimitiveTypeName expectedName) {
        PrimitiveType.PrimitiveTypeName actualName =
                descriptor.getPrimitiveType().getPrimitiveTypeName();
        checkArgument(
                actualName == expectedName,
                "Expected type name: %s, actual type name: %s",
                expectedName,
                actualName);
    }

    /** Reads `total` values from this columnReader into column. */
    @Override
    public final void readToVector(int readNumber, VECTOR vector) throws IOException {
        int rowId = 0;
        WritableIntVector dictionaryIds = null;

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Inspect the file metadata (parquet-tools meta) for the offending row group and regenerate the file
2. Drop the affected column from the projection to skip the broken chunk
3. If produced by a third-party writer, report/upgrade that writer; if by Flink itself, upgrade Flink

ParquetFileReader r = ParquetFileReader.open(conf, path);
for (BlockMetaData b : r.getRowGroups()) {
    for (ColumnChunkMetaData c : b.getColumns()) {
        if (c.getValueCount() < 0) throw new IllegalStateException("Bad value count in chunk metadata");
    }
}

catch (IOException e) { if ("totalValueCount == 0".equals(e.getMessage())) { /* skip/quarantine bad split and regenerate the file */ } else throw e; }

When it happens

Trigger: Creating a column reader over a row group whose column chunk reports totalValueCount == 0 in the page-reader store.

Common situations: Files with corrupt row-group/column-chunk metadata; writers that emit zero-value chunks for columns never present in that row group; truncated files.

Related errors

• expecting more rows but reached last block. Read {} out of {
• could not decode the dictionary for {}
• could not read page {} in col {}
• Corrupted Parquet schema
• Unknown ColumnIO, %s

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/2aad17df8141314f. Report an issue: GitHub.