{"record":{"id":"ff70f2ba994b961e","repo":"pola-rs/polars","slug":"unimplemented-anonymousscan","errorCode":null,"errorMessage":"unimplemented: AnonymousScan","messagePattern":"unimplemented: AnonymousScan","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/polars-stream/src/physical_plan/lower_ir.rs","lineNumber":832,"sourceCode":"                            .python_scan()\n                            .expect(\"should be python scan\");\n\n                        python_dataset_scan_to_reader_builder(expanded_scan)\n                    },\n\n                    #[cfg(feature = \"scan_lines\")]\n                    FileScanIR::Lines { name: _ } => {\n                        Arc::new(crate::nodes::io_sources::lines::LineReaderBuilder {\n                            prefetch_limit: RelaxedCell::new_usize(0),\n                            prefetch_semaphore: std::sync::OnceLock::new(),\n                            shared_prefetch_wait_group_slot: Default::default(),\n                            io_metrics: std::sync::OnceLock::new(),\n                        }) as _\n                    },\n\n                    FileScanIR::ExpandedPaths { name: _ } => unreachable!(),\n\n                    FileScanIR::Anonymous { .. } => todo!(\"unimplemented: AnonymousScan\"),\n                };\n\n                {\n                    let cloud_options = unified_scan_args.cloud_options.clone().map(Arc::new);\n                    let file_schema = file_info.schema;\n\n                    let (projected_schema, file_schema) =\n                        multi_scan::functions::resolve_projections::resolve_projections(\n                            &output_schema,\n                            &file_schema,\n                            &mut hive_parts,\n                            unified_scan_args\n                                .row_index\n                                .as_ref()\n                                .map(|ri| ri.name.as_str()),\n                            unified_scan_args\n                                .include_file_paths\n                                .as_ref()","sourceCodeStart":814,"sourceCodeEnd":850,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/crates/polars-stream/src/physical_plan/lower_ir.rs#L814-L850","documentation":"The new streaming engine's IR lowering has no physical source node for anonymous scans (FileScanIR::Anonymous), so it panics with todo!(). Anonymous scans inject arbitrary user callbacks that the streaming graph cannot schedule; only concrete file/IO sources are lowered.","triggerScenarios":"Executing a lazy query with engine='streaming' (or a context where the streaming engine is default) whose source is an anonymous scan - in Python typically pl.scan_pyarrow_dataset(...) or another scan function backed by a user callback - at collect()/explain() time.","commonSituations":"Users switch queries to the streaming engine (large data, low memory) while keeping a pyarrow-dataset or custom-callback scan; or polars automatically falls back to an anonymous scan for a late-materialized DataFrame.","solutions":["Run the query with the in-memory engine: lf.collect(engine='in-memory')","Materialize the source first (pl.read_* into a DataFrame, then pl.LazyFrame(df)) so streaming only handles downstream stages","Replace the anonymous scan with a native scan of concrete files (scan_parquet/scan_csv/scan_ipc) that the streaming engine supports"],"exampleFix":"# before\nlf = pl.scan_pyarrow_dataset(ds)  # anonymous scan\nlf.collect(engine=\"streaming\")  # panics during IR lowering\n\n# after\nlf.collect(engine=\"in-memory\")  # in-memory engine lowers anonymous scans","handlingStrategy":"fallback","validationCode":"def is_anonymous_scan(lf: pl.LazyFrame) -> bool:\n    return \"scan_pyarrow_dataset\" in (lf.explain() or \"\") or \" Anonymous\" in lf.explain()","typeGuard":"def engine_for(lf: pl.LazyFrame, preferred: str = \"streaming\") -> str:\n    try:\n        lf.collect(engine=preferred)\n        return preferred\n    except Exception:\n        return \"in-memory\"","tryCatchPattern":"try:\n    out = lf.collect(engine=\"streaming\")\nexcept pl.exceptions.PanicException:\n    out = lf.collect(engine=\"in-memory\")  # anonymous scans need the in-memory engine","preventionTips":["Avoid scan_pyarrow_dataset and callback-based scans when targeting the streaming engine","Prefer native scan_parquet/scan_csv/scan_ipc sources for streaming pipelines","Materialize custom sources to DataFrames before handing data to streaming queries"],"tags":["rust","polars","panic","todo","streaming","anonymous-scan","engine","lazyframe"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}