{"id":"32dea24e3ace4700","repo":"pypa/pip","slug":"contains-invalid-locale-encoding-characters","errorCode":null,"errorMessage":"contains invalid {locale_encoding} characters","messagePattern":"contains invalid (.+?) characters","errorType":"exception","errorClass":"ConfigurationFileCouldNotBeLoaded","httpStatus":null,"severity":"error","filePath":"src/pip/_internal/configuration.py","lineNumber":300,"sourceCode":"            items = parser.items(section)\n            self._config[variant].setdefault(fname, {})\n            self._config[variant][fname].update(self._normalized_keys(section, items))\n\n        return parser\n\n    def _construct_parser(self, fname: str) -> RawConfigParser:\n        parser = configparser.RawConfigParser()\n        # If there is no such file, don't bother reading it but create the\n        # parser anyway, to hold the data.\n        # Doing this is useful when modifying and saving files, where we don't\n        # need to construct a parser.\n        if os.path.exists(fname):\n            locale_encoding = get_locale_encoding()\n            try:\n                parser.read(fname, encoding=locale_encoding)\n            except UnicodeDecodeError:\n                # See https://github.com/pypa/pip/issues/4963\n                raise ConfigurationFileCouldNotBeLoaded(\n                    reason=f\"contains invalid {locale_encoding} characters\",\n                    fname=fname,\n                )\n            except configparser.Error as error:\n                # See https://github.com/pypa/pip/issues/4893\n                raise ConfigurationFileCouldNotBeLoaded(error=error)\n        return parser\n\n    def _load_environment_vars(self) -> None:\n        \"\"\"Loads configuration from environment variables\"\"\"\n        self._config[kinds.ENV_VAR].setdefault(\":env:\", {})\n        self._config[kinds.ENV_VAR][\":env:\"].update(\n            self._normalized_keys(\":env:\", self.get_environ_vars())\n        )\n\n    def _normalized_keys(\n        self, section: str, items: Iterable[tuple[str, Any]]\n    ) -> dict[str, Any]:","sourceCodeStart":282,"sourceCodeEnd":318,"githubUrl":"https://github.com/pypa/pip/blob/d7d0d0a39494e28ec1c407bd0680e4a4d1067791/src/pip/_internal/configuration.py#L282-L318","documentation":"Raised as ConfigurationFileCouldNotBeLoaded in Configuration._construct_parser() at configuration.py:296-303 when parser.read(fname, encoding=locale_encoding) raises UnicodeDecodeError. The config file contains bytes that are invalid under the detected locale encoding, so pip cannot safely parse it. The message reports the encoding that was in effect.","triggerScenarios":"A pip.conf/pip.ini file saved in an encoding incompatible with the system locale (e.g. UTF-16, or containing bytes outside the locale charset) being read at configuration.py:297. The UnicodeDecodeError is caught and re-raised as a structured ConfigurationFileCouldNotBeLoaded with the locale encoding named.","commonSituations":"On Windows a pip.ini saved as UTF-16 by PowerShell's Out-File; a config file edited on a different OS with a different charset; non-ASCII characters (e.g. in a URL or comment) saved in a charset the running locale does not cover; LANG/LC_ALL unset causing a narrow locale like ASCII.","solutions":["Re-save the config file as UTF-8 (the most portable choice): 'iconv -f UTF-16 -t UTF-8 pip.conf > pip.conf.utf8'.","Remove non-ASCII characters from the config file if they are not essential.","Set a UTF-8 locale in your shell (export LANG=en_US.UTF-8) and retry.","On Windows, re-save pip.ini with explicit UTF-8 encoding (e.g. Set-Content -Encoding utf8)."],"exampleFix":"# before — pip.ini saved as UTF-16 by PowerShell\n# fix encoding\niconv -f UTF-16 -t UTF-8 ~/.pip/pip.ini > /tmp/pip.conf && mv /tmp/pip.conf ~/.pip/pip.ini","handlingStrategy":"validation","validationCode":"import os\n\ndef config_file_is_utf8(path: str) -> bool:\n    if not os.path.exists(path):\n        return True\n    with open(path, 'rb') as f:\n        data = f.read()\n    try:\n        data.decode('utf-8')\n        return True\n    except UnicodeDecodeError:\n        return False\n\nif not config_file_is_utf8(os.path.expanduser('~/.pip/pip.conf')):\n    print('ERROR: config file is not valid UTF-8; re-save as UTF-8', flush=True)","typeGuard":"def is_valid_utf8_file(path: str) -> bool:\n    try:\n        with open(path, 'rb') as f:\n            f.read().decode('utf-8')\n        return True\n    except (UnicodeDecodeError, OSError):\n        return False","tryCatchPattern":"from pip._internal.exceptions import ConfigurationFileCouldNotBeLoaded\ntry:\n    cfg.load()\nexcept ConfigurationFileCouldNotBeLoaded as e:\n    if 'invalid' in str(e) and 'characters' in str(e):\n        # re-encode the offending file to UTF-8 and retry\n        import subprocess\n        subprocess.run(['iconv','-f','UTF-16','-t','UTF-8', e.fname, '-o', e.fname])\n        cfg.load()","preventionTips":["Save pip config files as UTF-8 (avoid UTF-16/UTF-32, especially on Windows PowerShell).","Set a UTF-8 locale (export LANG=en_US.UTF-8) in environments that read pip config.","Strip non-ASCII characters from config values and comments where possible.","Validate config file encoding with a pre-flight check before running pip."],"tags":["pip","config","configuration","encoding","unicode","locale"],"analyzedSha":"d7d0d0a39494e28ec1c407bd0680e4a4d1067791","analyzedAt":"2026-08-04T20:55:04.259Z","schemaVersion":2}