{"record":{"id":"0d4bef7fbbb1c414","repo":"pola-rs/polars","slug":"pre-slice","errorCode":null,"errorMessage":"{pre_slice:?}","messagePattern":"\\{pre_slice:\\?\\}","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/polars-stream/src/nodes/io_sources/csv/mod.rs","lineNumber":198,"sourceCode":"                    row_position_on_end_tx,\n                },\n        } = args\n        else {\n            panic!(\"unsupported args: {:?}\", args)\n        };\n\n        assert!(row_index.is_none()); // Handled outside the reader for now.\n\n        match &pre_slice {\n            Some(Slice::Negative { .. }) => unimplemented!(),\n\n            // We don't account for comments when slicing lines. We should never hit this panic -\n            // the FileReaderBuilder does not indicate PRE_SLICE support when we have a comment\n            // prefix.\n            Some(pre_slice)\n                if self.options.parse_options.comment_prefix.is_some() && pre_slice.len() > 0 =>\n            {\n                panic!(\"{pre_slice:?}\")\n            },\n\n            _ => {},\n        }\n\n        // There are two byte sourcing strategies `ReaderSource`: (a) async parallel prefetch using a\n        // streaming pipeline, or (b) memory-mapped, only to be used for uncompressed local files.\n        // The `compressed_reader` (of type `ByteSourceReader`) abstracts these source types.\n        // The `use_async_prefetch` flag controls the optional pipeline startup behavior.\n        let use_async_prefetch =\n            !(matches!(byte_source.as_ref(), &DynByteSource::Buffer(_)) && compression.is_none());\n\n        const ASSUMED_COMPRESSION_RATIO: usize = 4;\n        let decompressed_file_size_hint = match compression {\n            None => Some(file_size),\n            Some(_) => Some(file_size * ASSUMED_COMPRESSION_RATIO),\n        };\n","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/pola-rs/polars/blob/fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e/crates/polars-stream/src/nodes/io_sources/csv/mod.rs#L180-L216","documentation":"The streaming CSV reader cannot account for comment prefixes while performing line slicing. If a pre-slice is requested on a CSV whose parse options define a comment_prefix, begin_read panics with the slice value for diagnosis. The comment notes this should be unreachable because the FileReaderBuilder does not advertise PRE_SLICE support when a comment prefix is set — hitting it signals an internal capability-planning bug.","triggerScenarios":"A streaming CSV scan with comment_prefix configured (e.g. comment_prefix='#') where the planner still pushes a non-empty pre_slice into the reader's args — an internal invariant violation between capability declaration and slice pushdown.","commonSituations":"Reading comment-annotated CSVs with optimizations that pre-skip lines (e.g. skipping to a byte/line offset); version combinations where the builder/reader capability contract regressed.","solutions":["Remove or disable comment_prefix for the scan (pre-filter comments after reading) so the pre-slice path can proceed.","Upgrade Polars; this is a builder/reader contract bug likely fixed upstream.","Disable the new streaming engine for this scan as a workaround.","Report the query and slice value printed in the panic to the maintainers."],"exampleFix":"// before\nLazyCsvReader::new(path).with_comment_prefix(Some(\"#\")).finish().slice(1000, 10).collect_streaming()\n// after\nLazyCsvReader::new(path).finish().slice(1000, 10).collect_streaming() // filter '#' lines post-read if needed","handlingStrategy":"fallback","validationCode":"# If you set comment_prefix on a CSV scan, avoid triggering line pre-slicing:\nassert not (scan_has_comment_prefix and plan_uses_pre_slice), \"combination unsupported in streaming CSV reader\"","typeGuard":"def safe_combo(comment_prefix: str | None, pre_slice: tuple | None) -> bool:\n    return comment_prefix is None or pre_slice is None","tryCatchPattern":"try:\n    df = lf_with_comments.collect(engine=\"streaming\")\nexcept pl.exceptions.PolarsPanicError:\n    df = lf_no_comment_prefix.collect(engine=\"in-memory\")  # then filter comment rows manually","preventionTips":["Don't combine comment_prefix with queries that rely on line-offset pre-slicing","Strip comments post-read instead of via comment_prefix when slicing is involved","Upgrade polars if you hit this — it indicates a builder/reader capability bug","Report the exact query; the panic message includes the offending slice"],"tags":["rust","panic","csv","internal-invariant","streaming"],"backgroundTag":"internal-invariant-violation","analyzedSha":"fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e","analyzedAt":"2026-09-18T22:14:11.667Z","contentChangedAt":"2026-09-18T22:14:11.667Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}