{"record":{"id":"c055c88abd7c830c","repo":"pola-rs/polars","slug":"invalid-value-for-polars-force-csv-infer-read-size","errorCode":null,"errorMessage":"invalid value for POLARS_FORCE_CSV_INFER_READ_SIZE: {x}","messagePattern":"invalid value for POLARS_FORCE_CSV_INFER_READ_SIZE: (.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/polars-io/src/csv/read/streaming.rs","lineNumber":409,"sourceCode":"/// Iterate over valid CSV lines produced by reader.\n///\n/// Returning `ConsumeDiscard` after `ConsumeKeep` is a logic error, since a segmented `Buffer`\n/// can't be constructed.\nfn for_each_line_from_reader_from_compressed_reader(\n    parse_options: &CsvParseOptions,\n    is_file_start: bool,\n    mut prev_leftover: Buffer<u8>,\n    initial_read_size: usize,\n    reader: &mut CompressedReader,\n    mut line_fn: impl FnMut(Buffer<u8>) -> PolarsResult<LineUse>,\n) -> PolarsResult<Buffer<u8>> {\n    let mut is_first_line = is_file_start;\n\n    let fixed_read_size = std::env::var(\"POLARS_FORCE_CSV_INFER_READ_SIZE\")\n        .map(|x| {\n            x.parse::<NonZeroUsize>()\n                .unwrap_or_else(|_| {\n                    panic!(\"invalid value for POLARS_FORCE_CSV_INFER_READ_SIZE: {x}\")\n                })\n                .get()\n        })\n        .ok();\n\n    let mut read_size = fixed_read_size.unwrap_or(initial_read_size);\n    let mut retain_offset = None;\n\n    loop {\n        let (mut slice, bytes_read) = reader.read_next_slice(&prev_leftover, read_size)?;\n        if slice.is_empty() {\n            return Ok(Buffer::new());\n        }\n\n        if is_first_line {\n            is_first_line = false;\n            const UTF8_BOM_MARKER: Option<&[u8]> = Some(b\"\\xef\\xbb\\xbf\");\n            if slice.get(0..3) == UTF8_BOM_MARKER {","sourceCodeStart":391,"sourceCodeEnd":427,"githubUrl":"https://github.com/pola-rs/polars/blob/5d8ebabf11caea54a5c29178a64a058762f49766/crates/polars-io/src/csv/read/streaming.rs#L391-L427","documentation":"POLARS_FORCE_CSV_INFER_READ_SIZE is an internal knob (used mainly by polars' own tests) that pins a fixed chunk size for the streaming reader during CSV schema inference, overriding the adaptive initial_read_size. The value must parse as a NonZeroUsize; this panic fires inside the CompressedReader streaming loop when it does not. The variable is read on every call, so the panic hits as soon as inference streams the first rows.","triggerScenarios":"The variable leaks into an environment (left over from debugging polars, copied from polars' test scripts or CI env) set to 0, a non-number, or a quoted/empty string, and any read_csv with schema inference runs through the compressed-reader path.","commonSituations":"Docker images or CI presets that baked debug POLARS_* variables; shell rc files carrying experiment leftovers; running with polars' own test env in a production container.","solutions":["unset POLARS_FORCE_CSV_INFER_READ_SIZE - inference then uses its normal adaptive sizing","Or set it to a positive byte count, e.g. export POLARS_FORCE_CSV_INFER_READ_SIZE=8192","Audit Dockerfiles/CI env blocks for leftover POLARS_FORCE_* debug variables"],"exampleFix":"# before\nexport POLARS_FORCE_CSV_INFER_READ_SIZE=0   # panics during read_csv inference\n\n# after\nunset POLARS_FORCE_CSV_INFER_READ_SIZE","handlingStrategy":"validation","validationCode":"fn assert_no_debug_csv_env() -> Result<(), String> {\n    if let Ok(s) = std::env::var(\"POLARS_FORCE_CSV_INFER_READ_SIZE\") {\n        if s.parse::<std::num::NonZeroUsize>().is_err() {\n            return Err(format!(\"POLARS_FORCE_CSV_INFER_READ_SIZE must be a non-zero integer, got {s:?} - it is a polars-internal test knob, prefer unsetting it\"));\n        }\n    }\n    Ok(())\n}","typeGuard":null,"tryCatchPattern":"Deterministic env panic inside the inference loop; catch_unwind can only re-label it. Unset or fix the variable instead.","preventionTips":["Treat POLARS_FORCE_* as test-only: scrub them from production images","Assert on startup that no POLARS_FORCE_* variables are set","Keep debug env experiments in throwaway shells, never in shared .env files"],"tags":["polars","csv","environment-variable","streaming","inference","testing","panic"],"backgroundTag":"invalid-env-var-value","analyzedSha":"5d8ebabf11caea54a5c29178a64a058762f49766","analyzedAt":"2026-08-19T12:15:06.350Z","contentChangedAt":"2026-08-19T12:15:06.350Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}