{"record":{"id":"7e078468d6762a15","repo":"pandas-dev/pandas","slug":"disallowed-deserialization-of-arrow-py-extension","errorCode":null,"errorMessage":"Disallowed deserialization of 'arrow.py_extension_type':\nstorage_type = {storage_type}\nserialized = {serialized}\npickle disassembly:\n{pickle_disassembly}\n\nReading of untrusted Parquet or Feather files with a PyExtensionType column\nallows arbitrary code execution.\nIf you trust this file, you can enable reading the extension type by one of:\n\n- upgrading to pyarrow >= 14.0.1, and call `pa.PyExtensionType.set_auto_load(True)`\n- install pyarrow-hotfix (`pip install pyarrow-hotfix`) and disable it by running\n  `import pyarrow_hotfix; pyarrow_hotfix.uninstall()`\n\nWe strongly recommend updating your Parquet/Feather files to use extension types\nderived from `pyarrow.ExtensionType` instead, and register this type explicitly.\n","messagePattern":"Disallowed deserialization of 'arrow\\.py_extension_type':\nstorage_type = \\{storage_type\\}\nserialized = \\{serialized\\}\npickle disassembly:\n\\{pickle_disassembly\\}\n\nReading of untrusted Parquet or Feather files with a PyExtensionType column\nallows arbitrary code execution\\.\nIf you trust this file, you can enable reading the extension type by one of:\n\n- upgrading to pyarrow >= 14\\.0\\.1, and call `pa\\.PyExtensionType\\.set_auto_load\\(True\\)`\n- install pyarrow-hotfix \\(`pip install pyarrow-hotfix`\\) and disable it by running\n  `import pyarrow_hotfix; pyarrow_hotfix\\.uninstall\\(\\)`\n\nWe strongly recommend updating your Parquet/Feather files to use extension types\nderived from `pyarrow\\.ExtensionType` instead, and register this type explicitly\\.\n","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"pandas/core/arrays/arrow/extension_types.py","lineNumber":158,"sourceCode":"    if not pa_version_under14p1:\n        return\n\n    # if https://github.com/pitrou/pyarrow-hotfix was installed and enabled\n    if getattr(pyarrow, \"_hotfix_installed\", False):\n        return\n\n    class ForbiddenExtensionType(pyarrow.ExtensionType):\n        def __arrow_ext_serialize__(self) -> bytes:\n            return b\"\"\n\n        @classmethod\n        def __arrow_ext_deserialize__(cls, storage_type, serialized):\n            import io\n            import pickletools\n\n            out = io.StringIO()\n            pickletools.dis(serialized, out)\n            raise RuntimeError(\n                _ERROR_MSG.format(\n                    storage_type=storage_type,\n                    serialized=serialized,\n                    pickle_disassembly=out.getvalue(),\n                )\n            )\n\n    pyarrow.unregister_extension_type(\"arrow.py_extension_type\")\n    pyarrow.register_extension_type(\n        ForbiddenExtensionType(pyarrow.null(), \"arrow.py_extension_type\")\n    )\n\n    pyarrow._hotfix_installed = True\n\n\npatch_pyarrow()\n","sourceCodeStart":140,"sourceCodeEnd":175,"githubUrl":"https://github.com/pandas-dev/pandas/blob/71959b8cb9b2459c16e14b34f28b178ccfe14735/pandas/core/arrays/arrow/extension_types.py#L140-L175","documentation":"Raised by the ForbiddenExtensionType injected by pandas.core.arrays.arrow.extension_types.patch_pyarrow() when deserializing a column whose pyarrow extension type is 'arrow.py_extension_type' on pyarrow < 14.0.1 (without pyarrow-hotfix). This is the pandas-side mitigation for CVE-2023-47248: deserializing a PyExtensionType column from Parquet/Feather executes arbitrary pickle code, so pandas blocks it with a RuntimeError that includes a pickle disassembly and remediation steps. It fires only on the old, vulnerable pyarrow versions; pyarrow >= 14.0.1 has its own auto-load gate.","triggerScenarios":"Calling pd.read_parquet / pd.read_feather on a file that contains an `arrow.py_extension_type` column, while running pyarrow < 14.0.1 and without pyarrow-hotfix installed/enabled. The file may be untrusted or simply produced by an older writer that used PyExtensionType.","commonSituations":"CI/CD pinned to an old pyarrow (<14.0.1); reading Parquet/Feather written by legacy pyarrow or third-party tools that registered PyExtensionType; shared datasets downloaded from untrusted sources. Security-sensitive environments (regulated, multi-tenant) must treat this as an explicit block, not a nuisance.","solutions":["Upgrade pyarrow to >= 14.0.1 (preferred); then optionally enable trusted loading via pa.PyExtensionType.set_auto_load(True).","If you must stay on old pyarrow, install pyarrow-hotfix (`pip install pyarrow-hotfix`) for general protection; to read a TRUSTED file, run `import pyarrow_hotfix; pyarrow_hotfix.uninstall()` (understand the risk first).","Rewrite the source Parquet/Feather file using extension types derived from pyarrow.ExtensionType and register them explicitly instead of PyExtensionType.","If the file is untrusted, do NOT bypass the guard; treat it as untrusted input and sanitize at ingestion."],"exampleFix":"# before (triggers on pyarrow < 14.0.1)\ndf = pd.read_parquet(\"legacy_with_pyext.parquet\")  # RuntimeError\n\n# after (preferred): upgrade pyarrow then optionally allow trusted loading\n# pip install -U \"pyarrow>=14.0.1\"\nimport pyarrow as pa\npa.PyExtensionType.set_auto_load(True)\ndf = pd.read_parquet(\"legacy_with_pyext.parquet\")\n\n# after (legacy pyarrow, trusted file only):\n# pip install pyarrow-hotfix\nimport pyarrow_hotfix\npyarrow_hotfix.uninstall()\ndf = pd.read_parquet(\"legacy_with_pyext.parquet\")","handlingStrategy":"validation","validationCode":"import pyarrow as pa\nfrom packaging.version import Version\n\ndef safe_read_parquet(path):\n    if Version(pa.__version__) < Version(\"14.0.1\"):\n        # check the file's schema for arrow.py_extension_type columns before reading\n        schema = pa.parquet.read_schema(path) if hasattr(pa, \"parquet\") else None\n        if schema is not None and \"py_extension_type\" in str(schema):\n            raise RuntimeError(\n                \"Refusing to read file with PyExtensionType column on pyarrow < 14.0.1 \"\n                \"(CVE-2023-47248). Upgrade pyarrow or only bypass with a TRUSTED file.\"\n            )\n    return pd.read_parquet(path)","typeGuard":"import pyarrow as pa\nfrom packaging.version import Version\n\ndef is_safe_pyarrow_version() -> bool:\n    try:\n        from packaging.version import Version\n        return Version(pa.__version__) >= Version(\"14.0.1\")\n    except Exception:\n        return False","tryCatchPattern":"try:\n    df = pd.read_parquet(path)\nexcept RuntimeError as e:\n    if \"arrow.py_extension_type\" in str(e):\n        # only for TRUSTED files; upgrade pyarrow or rewrite file instead\n        import pyarrow_hotfix\n        pyarrow_hotfix.uninstall()\n        df = pd.read_parquet(path)\n    else:\n        raise","preventionTips":["Pin pyarrow >= 14.0.1 in all environments (CI, prod, notebooks).","Install pyarrow-hotfix as defense-in-depth on legacy stacks.","Never bypass the deserialization guard on untrusted files; sanitize at ingestion.","Prefer extension types derived from pyarrow.ExtensionType over PyExtensionType when writing files.","Scan downloaded Parquet/Feather files for 'arrow.py_extension_type' columns before reading."],"tags":["security","pyarrow","deserialization","cve","parquet","pickle"],"analyzedSha":"71959b8cb9b2459c16e14b34f28b178ccfe14735","analyzedAt":"2026-08-07T01:30:20.476Z","schemaVersion":2},"datasetVersion":"2026-08-07T03:17:09.362Z"}