{"record":{"id":"dd65a60017d80868","repo":"pola-rs/polars","slug":"pre-slice-reader-starter","errorCode":null,"errorMessage":"{pre_slice:?} {}","messagePattern":"\\{pre_slice:\\?\\} \\{\\}","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/polars-stream/src/nodes/io_sources/multi_scan/pipeline/tasks/reader_starter.rs","lineNumber":153,"sourceCode":"            let external_filter_mask = if let Some(row_deletions) = row_deletions {\n                Some(row_deletions.into_external_filter_mask().await?)\n            } else {\n                None\n            };\n\n            // * This translates the `pre_slice` to physical offsets (i.e. apply before deleting rows).\n            //   The slice remains the same if there are no row deletions.\n            let pre_slice_this_file: Option<PhysicalSlice> =\n                pre_slice_this_file.map(|pre_slice| match pre_slice {\n                    Slice::Positive { .. } => {\n                        PhysicalSlice::new(pre_slice, external_filter_mask.as_ref())\n                    },\n\n                    // This is hit here for NDJSON single file negative slice, we just passthrough\n                    // in this case.\n                    Slice::Negative { .. } => {\n                        if external_filter_mask.is_some() {\n                            unimplemented!(\n                                \"{pre_slice:?} {}\",\n                                ExternalFilterMask::log_display(external_filter_mask.as_ref())\n                            )\n                        }\n\n                        assert!(\n                            (extra_ops.row_index.is_none()\n                                || reader_capabilities.contains(ReaderCapabilities::ROW_INDEX))\n                                && (external_filter_mask.is_none()\n                                    || reader_capabilities\n                                        .contains(ReaderCapabilities::EXTERNAL_FILTER_MASK))\n                        );\n\n                        PhysicalSlice {\n                            slice: pre_slice,\n                            // Note, this is not the correct starting position. The assertion above\n                            // should ensure this value is not used in post-apply.\n                            slice_start_position: RowCounter::default(),","sourceCodeStart":135,"sourceCodeEnd":171,"githubUrl":"https://github.com/pola-rs/polars/blob/fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e/crates/polars-stream/src/nodes/io_sources/multi_scan/pipeline/tasks/reader_starter.rs#L135-L171","documentation":"The multi-scan reader starter hits an unimplemented!() when a multi-file scan needs to pass through a negative Slice (pre_slice) while an ExternalFilterMask is present. Negative slices are only supported in this code path for the NDJSON single-file case without external filtering; combining a negative slice with an external filter mask is not implemented.","triggerScenarios":"A streaming multi-scan (e.g. NDJSON/multi-file source) whose pre_slice is Slice::Negative while an external filter mask (from predicates pushed into the scan) is also present, in run().","commonSituations":"Calling scan_ndjson / multi-file scan with slice(..., negative offset) combined with a filter predicate, e.g. df.filter(...).slice(-5, 5) pushed down to a streaming NDJSON scan on a recent Polars version.","solutions":["Avoid combining a negative slice with a pushed-down filter on this source: materialize the filter first and slice afterwards, or collect then slice.","Replace the negative slice with an equivalent positive offset (compute length first) so it doesn't route through the unimplemented path.","Check Polars issue tracker/upgrades: this is an explicit unimplemented path and may gain support in newer versions."],"exampleFix":"// before\ndf = pl.scan_ndjson(path).filter(pl.col('a') > 1).slice(-5, 5).collect()\n// after\ndf = pl.scan_ndjson(path).filter(pl.col('a') > 1).collect().slice(-5, 5)","handlingStrategy":"validation","validationCode":"# Python: avoid negative slices combined with pushed-down filters on streaming scans\nif offset < 0 and predicate is not None:\n    raise ValueError('Negative slice with filter pushdown is not implemented for this streaming source; materialize first')","typeGuard":"def is_positive_slice(offset, length):\n    return offset is not None and offset >= 0","tryCatchPattern":"try:\n    df = lf.filter(pred).slice(offset, length).collect()\nexcept Exception as e:\n    if 'unimplemented' in str(e) and 'pre_slice' in str(e):\n        df = lf.collect().slice(offset, length)\n    else:\n        raise","preventionTips":["Don't combine negative slice() offsets with filter predicates in lazy/streaming scans.","Convert negative offsets to positive ones using a known row count before pushing to the engine.","Check the Polars changelog for when negative-slice + filter pushdown becomes supported."],"tags":["rust","unimplemented","slice","ndjson","streaming"],"backgroundTag":"unsupported-operation","analyzedSha":"fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e","analyzedAt":"2026-09-18T22:14:11.667Z","contentChangedAt":"2026-09-18T22:14:11.667Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}