{"record":{"id":"a756e1abe66a419d","repo":"pola-rs/polars","slug":"fsspec-is-required-for-storage-options-argumen","errorCode":null,"errorMessage":"`fsspec` is required for `storage_options` argument","messagePattern":"`fsspec` is required for `storage_options` argument","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/_utils.py","lineNumber":169,"sourceCode":"    An http URL is read into a buffer and returned as a `BytesIO`.\n\n    When `encoding` is not `utf8` or `utf8-lossy`, the whole file is\n    first read in Python and decoded using the specified encoding and\n    returned as a `BytesIO` (for usage with `read_csv`). If encoding\n    ends with \"-lossy\", characters that can't be decoded are replaced\n    with `�`.\n\n    A `bytes` file is returned as a `BytesIO` if `use_pyarrow=True`.\n\n    When fsspec is installed, remote file(s) is (are) opened with\n    `fsspec.open(file, **kwargs)` or `fsspec.open_files(file, **kwargs)`.\n    If encoding is not `utf8` or `utf8-lossy`, decoding is handled by\n    fsspec too.\n    \"\"\"\n    storage_options = storage_options.copy() if storage_options else {}\n    if storage_options and not _FSSPEC_AVAILABLE:\n        msg = \"`fsspec` is required for `storage_options` argument\"\n        raise ImportError(msg)\n\n    # Small helper to use a variable as context\n    @contextmanager\n    def managed_file(file: Any) -> Iterator[Any]:\n        try:\n            yield file\n        finally:\n            pass\n\n    has_utf8_utf8_lossy_encoding = (\n        encoding in {\"utf8\", \"utf8-lossy\"} if encoding else True\n    )\n    encoding_str = encoding if encoding else \"utf8\"\n    encoding_str, encoding_errors = (\n        (encoding_str[:-6], \"replace\")\n        if encoding_str.endswith(\"-lossy\")\n        else (encoding_str, \"strict\")\n    )","sourceCodeStart":151,"sourceCodeEnd":187,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/io/_utils.py#L151-L187","documentation":"Eager readers (read_csv, read_parquet, ...) route source preparation through process_file, which opens remote paths with fsspec whenever `storage_options` is supplied (py-polars/src/polars/io/_utils.py:166-169). If the dict is non-empty and fsspec is not importable, an ImportError is raised before anything is read. The lazy scan_* APIs use polars' native Rust cloud stack and do not go through this check.","triggerScenarios":"pl.read_csv('s3://bucket/f.csv', storage_options={'anon': 'true'}) with fsspec not installed; passing storage_options even for a local file triggers it, because the check runs on any non-empty dict.","commonSituations":"Slim Docker/CI images that installed polars without the fssesspec/s3fs/gcsfs extras; code migrated from scan_csv (native cloud support) to read_csv; adding storage_options to an existing job on an environment that never had fsspec.","solutions":["Install fsspec plus the protocol extra: pip install fsspec s3fs (for s3://) or gcsfs (for gs://)","Switch to the lazy API pl.scan_csv(...)/pl.scan_parquet(...), whose native credential handling needs no fsspec","Drop storage_options if environment defaults (env vars, instance profiles) already provide access"],"exampleFix":"# before (ImportError: `fsspec` is required)\ndf = pl.read_csv(\"s3://bucket/f.csv\", storage_options={\"anon\": \"true\"})\n# after (native cloud path, no fsspec needed)\ndf = pl.scan_csv(\"s3://bucket/f.csv\").collect()","handlingStrategy":"validation","validationCode":"import importlib.util\n\ndef can_use_storage_options() -> bool:\n    return importlib.util.find_spec(\"fsspec\") is not None\n\nif opts and not can_use_storage_options():\n    raise SystemExit(\"storage_options requires fsspec: pip install fsspec s3fs\")","typeGuard":null,"tryCatchPattern":"try:\n    df = pl.read_csv(url, storage_options=opts)\nexcept ImportError as e:\n    if \"fsspec\" in str(e):\n        raise SystemExit(\"pip install fsspec (plus s3fs/gcsfs for your scheme)\") from e\n    raise","preventionTips":["Declare fsspec + protocol extras in environments that pass storage_options to eager readers","Prefer scan_* APIs for remote paths to use the native credential stack","Run a dependency check at startup when storage_options is part of the configuration"],"tags":["polars","io","fsspec","cloud","importerror","dependencies"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}