pola-rs/polars · error

{pre_slice:?}

Error message

{pre_slice:?}

What it means

The streaming CSV reader cannot account for comment prefixes while performing line slicing. If a pre-slice is requested on a CSV whose parse options define a comment_prefix, begin_read panics with the slice value for diagnosis. The comment notes this should be unreachable because the FileReaderBuilder does not advertise PRE_SLICE support when a comment prefix is set — hitting it signals an internal capability-planning bug.

Solutions

1. Remove or disable comment_prefix for the scan (pre-filter comments after reading) so the pre-slice path can proceed.
2. Upgrade Polars; this is a builder/reader contract bug likely fixed upstream.
3. Disable the new streaming engine for this scan as a workaround.
4. Report the query and slice value printed in the panic to the maintainers.

Example fix

// before
LazyCsvReader::new(path).with_comment_prefix(Some("#")).finish().slice(1000, 10).collect_streaming()
// after
LazyCsvReader::new(path).finish().slice(1000, 10).collect_streaming() // filter '#' lines post-read if needed

# If you set comment_prefix on a CSV scan, avoid triggering line pre-slicing:
assert not (scan_has_comment_prefix and plan_uses_pre_slice), "combination unsupported in streaming CSV reader"

def safe_combo(comment_prefix: str | None, pre_slice: tuple | None) -> bool:
    return comment_prefix is None or pre_slice is None

try:
    df = lf_with_comments.collect(engine="streaming")
except pl.exceptions.PolarsPanicError:
    df = lf_no_comment_prefix.collect(engine="in-memory")  # then filter comment rows manually

When it happens

Trigger: A streaming CSV scan with comment_prefix configured (e.g. comment_prefix='#') where the planner still pushes a non-empty pre_slice into the reader's args — an internal invariant violation between capability declaration and slice pushdown.

Common situations: Reading comment-annotated CSVs with optimizations that pre-skip lines (e.g. skipping to a byte/line offset); version combinations where the builder/reader capability contract regressed.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• unimplemented
• addition overflow: +
• activate dtype
• can only use timedelta string language with…
• cannot coerce datatypes

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/0d4bef7fbbb1c414. Report an issue: GitHub.

Appendix: source

Thrown at crates/polars-stream/src/nodes/io_sources/csv/mod.rs:198

                    row_position_on_end_tx,
                },
        } = args
        else {
            panic!("unsupported args: {:?}", args)
        };

        assert!(row_index.is_none()); // Handled outside the reader for now.

        match &pre_slice {
            Some(Slice::Negative { .. }) => unimplemented!(),

            // We don't account for comments when slicing lines. We should never hit this panic -
            // the FileReaderBuilder does not indicate PRE_SLICE support when we have a comment
            // prefix.
            Some(pre_slice)
                if self.options.parse_options.comment_prefix.is_some() && pre_slice.len() > 0 =>
            {
                panic!("{pre_slice:?}")
            },

            _ => {},
        }

        // There are two byte sourcing strategies `ReaderSource`: (a) async parallel prefetch using a
        // streaming pipeline, or (b) memory-mapped, only to be used for uncompressed local files.
        // The `compressed_reader` (of type `ByteSourceReader`) abstracts these source types.
        // The `use_async_prefetch` flag controls the optional pipeline startup behavior.
        let use_async_prefetch =
            !(matches!(byte_source.as_ref(), &DynByteSource::Buffer(_)) && compression.is_none());

        const ASSUMED_COMPRESSION_RATIO: usize = 4;
        let decompressed_file_size_hint = match compression {
            None => Some(file_size),
            Some(_) => Some(file_size * ASSUMED_COMPRESSION_RATIO),
        };

View on GitHub (pinned to fe841f959e)