{"record":{"id":"06dbde92fa730173","repo":"pola-rs/polars","slug":"python-function-failed","errorCode":null,"errorMessage":"python function failed","messagePattern":"python function failed","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/polars-python/src/lazyframe/general.rs","lineNumber":288,"sourceCode":"            .with_truncate_ragged_lines(truncate_ragged_lines)\n            .with_decimal_comma(decimal_comma)\n            .with_glob(glob)\n            .with_raise_if_empty(raise_if_empty)\n            .with_include_file_paths(include_file_paths.map(|x| x.into()))\n            .with_extra_columns_policy(extra_columns.0)\n            .with_missing_columns_policy(missing_columns.map(|x| x.0));\n\n        if let Some(new_columns) = new_columns {\n            r = r.with_column_names_overwrite(new_columns.0);\n        }\n\n        if let Some(lambda) = with_schema_modify {\n            let f = |schema: Schema| {\n                let iter = schema.iter_names().map(|s| s.as_str());\n                Python::attach(|py| {\n                    let names = PyList::new(py, iter).unwrap();\n\n                    let out = lambda.call1(py, (names,)).expect(\"python function failed\");\n                    let new_names = out\n                        .extract::<Vec<String>>(py)\n                        .expect(\"python function should return List[str]\");\n                    polars_ensure!(new_names.len() == schema.len(),\n                        ShapeMismatch: \"The length of the new names list should be equal to or less than the original column length\",\n                    );\n                    Ok(schema\n                        .iter_values()\n                        .zip(new_names)\n                        .map(|(dtype, name)| Field::new(name.into(), dtype.clone()))\n                        .collect())\n                })\n            };\n            r = r.with_schema_modify(f).map_err(PyPolarsErr::from)?\n        }\n\n        Ok(r.finish().map_err(PyPolarsErr::from)?.into())\n    }","sourceCodeStart":270,"sourceCodeEnd":306,"githubUrl":"https://github.com/pola-rs/polars/blob/fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e/crates/polars-python/src/lazyframe/general.rs#L270-L306","documentation":"When reading CSV with schema_modify (rename_columns via with_schema_modify), polars calls the user's Python lambda with the list of column names and expects a valid return. `expect(\"python function failed\")` panics if calling the lambda itself raised a Python exception, because the Rust side cannot gracefully convert a failure at this point into an error — it aborts with this message plus the original Python traceback context.","triggerScenarios":"pl.scan_csv(..., with_schema_modify=lambda names: ...) where the lambda raises any exception (NameError, TypeError, index errors, calling unsupported APIs) when invoked with the list of original column names.","commonSituations":"Lambdas referencing variables not in scope; calling string methods on the list instead of per-name; using functions like `names.lower()` (list has no such method); typos inside the renaming function.","solutions":["Fix the lambda so it does not raise: operate per-name, e.g. lambda names: [n.lower() for n in names].","Test the lambda standalone on a plain list of column names before using it in read/scan.","Wrap risky logic inside the lambda in try/except and return the original names on failure.","Upgrade polars if you need this failure surfaced as a Python exception instead of a panic."],"exampleFix":"# before\npl.scan_csv(path, with_schema_modify=lambda names: names.lower())  # AttributeError -> panic\n# after\npl.scan_csv(path, with_schema_modify=lambda names: [n.lower() for n in names])","handlingStrategy":"try-catch","validationCode":"fn = lambda names: [n.lower() for n in names]  # test first:\nassert isinstance(fn(['A', 'B']), list) and all(isinstance(x, str) for x in fn(['A', 'B']))","typeGuard":"def safe_renamer(fn):\n    def wrapped(names):\n        try:\n            return list(fn(names))\n        except Exception:\n            return names\n    return wrapped","tryCatchPattern":"try:\n    lf = pl.scan_csv(path, with_schema_modify=lambda names: [n.lower() for n in names])\n    df = lf.collect()\nexcept BaseException as e:\n    if 'python function failed' in str(e):\n        df = pl.read_csv(path)  # fallback: no renaming\n    else:\n        raise","preventionTips":["Unit-test rename lambdas on a plain list before use.","Keep the lambda pure and exception-free.","Return a list of str, one per input name."],"tags":["python","callback","csv","panic","lazyframe"],"backgroundTag":"invalid-argument-value","analyzedSha":"fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e","analyzedAt":"2026-09-18T22:14:11.667Z","contentChangedAt":"2026-09-18T22:14:11.667Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}