pola-rs/polars · error

{pre_slice:?} {}

Error message

{pre_slice:?} {}

What it means

The multi-scan reader starter hits an unimplemented!() when a multi-file scan needs to pass through a negative Slice (pre_slice) while an ExternalFilterMask is present. Negative slices are only supported in this code path for the NDJSON single-file case without external filtering; combining a negative slice with an external filter mask is not implemented.

Solutions

  1. Avoid combining a negative slice with a pushed-down filter on this source: materialize the filter first and slice afterwards, or collect then slice.
  2. Replace the negative slice with an equivalent positive offset (compute length first) so it doesn't route through the unimplemented path.
  3. Check Polars issue tracker/upgrades: this is an explicit unimplemented path and may gain support in newer versions.

Example fix

// before
df = pl.scan_ndjson(path).filter(pl.col('a') > 1).slice(-5, 5).collect()
// after
df = pl.scan_ndjson(path).filter(pl.col('a') > 1).collect().slice(-5, 5)
Defensive patterns

Strategy: validation

Validate before calling

# Python: avoid negative slices combined with pushed-down filters on streaming scans
if offset < 0 and predicate is not None:
    raise ValueError('Negative slice with filter pushdown is not implemented for this streaming source; materialize first')

Type guard

def is_positive_slice(offset, length):
    return offset is not None and offset >= 0

Try / catch

try:
    df = lf.filter(pred).slice(offset, length).collect()
except Exception as e:
    if 'unimplemented' in str(e) and 'pre_slice' in str(e):
        df = lf.collect().slice(offset, length)
    else:
        raise

Prevention

When it happens

Trigger: A streaming multi-scan (e.g. NDJSON/multi-file source) whose pre_slice is Slice::Negative while an external filter mask (from predicates pushed into the scan) is also present, in run().

Common situations: Calling scan_ndjson / multi-file scan with slice(..., negative offset) combined with a filter predicate, e.g. df.filter(...).slice(-5, 5) pushed down to a streaming NDJSON scan on a recent Polars version.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/dd65a60017d80868. Report an issue: GitHub.

Appendix: source

Thrown at crates/polars-stream/src/nodes/io_sources/multi_scan/pipeline/tasks/reader_starter.rs:153

            let external_filter_mask = if let Some(row_deletions) = row_deletions {
                Some(row_deletions.into_external_filter_mask().await?)
            } else {
                None
            };

            // * This translates the `pre_slice` to physical offsets (i.e. apply before deleting rows).
            //   The slice remains the same if there are no row deletions.
            let pre_slice_this_file: Option<PhysicalSlice> =
                pre_slice_this_file.map(|pre_slice| match pre_slice {
                    Slice::Positive { .. } => {
                        PhysicalSlice::new(pre_slice, external_filter_mask.as_ref())
                    },

                    // This is hit here for NDJSON single file negative slice, we just passthrough
                    // in this case.
                    Slice::Negative { .. } => {
                        if external_filter_mask.is_some() {
                            unimplemented!(
                                "{pre_slice:?} {}",
                                ExternalFilterMask::log_display(external_filter_mask.as_ref())
                            )
                        }

                        assert!(
                            (extra_ops.row_index.is_none()
                                || reader_capabilities.contains(ReaderCapabilities::ROW_INDEX))
                                && (external_filter_mask.is_none()
                                    || reader_capabilities
                                        .contains(ReaderCapabilities::EXTERNAL_FILTER_MASK))
                        );

                        PhysicalSlice {
                            slice: pre_slice,
                            // Note, this is not the correct starting position. The assertion above
                            // should ensure this value is not used in post-apply.
                            slice_start_position: RowCounter::default(),

View on GitHub (pinned to fe841f959e)