pola-rs/polars · error

We should have num_rows when we are writing

Error message

We should have num_rows when we are writing

What it means

Expect panic in write_page(): the compressed page reports no row count. Nested encodings derive num_rows from parent context; a None here means the page was produced without row information, which the invariant of write_page treats as a writer bug rather than a user error.

Source

Thrown at crates/polars-parquet/src/parquet/write/page.rs:63

    pub num_values: usize,
    /// The number of actual rows. For non-nested values, this is equal to the number of values.
    pub num_rows: usize,
    pub header_size: u64,
    pub offset: u64,
    pub bytes_written: u64,
    pub compression: Compression,
    pub statistics: Option<Statistics>,
}

pub fn write_page<W: Write>(
    writer: &mut W,
    offset: u64,
    compressed_page: &CompressedPage,
) -> ParquetResult<PageWriteSpec> {
    let num_values = compressed_page.num_values();
    let num_rows = compressed_page
        .num_rows()
        .expect("We should have num_rows when we are writing");

    let header = match &compressed_page {
        CompressedPage::Data(compressed_page) => assemble_data_page_header(compressed_page),
        CompressedPage::Dict(compressed_page) => assemble_dict_page_header(compressed_page),
    }?;

    let header_size = write_page_header(writer, &header)?;
    let mut bytes_written = header_size;

    bytes_written += match &compressed_page {
        CompressedPage::Data(compressed_page) => {
            writer.write_all(&compressed_page.buffer)?;
            compressed_page.buffer.len() as u64
        },
        CompressedPage::Dict(compressed_page) => {
            writer.write_all(&compressed_page.buffer)?;
            compressed_page.buffer.len() as u64
        },

View on GitHub (pinned to 9b5d73fd00)

Solutions

1. Internal invariant: num_rows must be known when writing the page; pass the row count through the write options.
2. Upgrade polars-parquet if this occurs with standard usage.

When it happens

Trigger: This panic/expect fires when execution reaches an unguarded state described by: "We should have num_rows when we are writing". Typical triggers: unsupported dtype or feature-gated code path reached at runtime, invalid user input or environment variable value, calling API methods in the wrong order or on mismatched types, or data (lengths, offsets, ranges) violating the function's preconditions.

Common situations: Encountered when input data or configuration does not meet the preconditions of the throwing code path ("We should have num_rows when we are writing"). Common cases: missing Cargo feature flags, mismatched dtypes/lengths between arrays or series, out-of-range temporal values, malformed environment variables, or operations on unsupported/complex nested types.

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/f670fbac94c3e5a9. Report an issue: GitHub.