{"record":{"id":"0345c9231b4a1efc","repo":"LykosAI/StabilityMatrix","slug":"pyvenv-cfg-is-utf-16-encoded-expected-utf-8-ascii-path","errorCode":null,"errorMessage":"pyvenv.cfg is UTF-16 encoded; expected UTF-8/ASCII: {path}","messagePattern":"pyvenv\\.cfg is UTF-16 encoded; expected UTF-8/ASCII: (.+?)","errorType":"exception","errorClass":"InvalidDataException","httpStatus":null,"severity":"error","filePath":"StabilityMatrix.Core/Python/PyVenvCfg.cs","lineNumber":63,"sourceCode":"        return new PyVenvCfg(entries);\n    }\n\n    /// <summary>\n    /// Loads a pyvenv.cfg file. Fails loudly on non-UTF-8 encodings instead of\n    /// silently mangling the file.\n    /// </summary>\n    public static PyVenvCfg Load(string path)\n    {\n        var bytes = File.ReadAllBytes(path);\n\n        // pyvenv.cfg is UTF-8/ASCII; reject UTF-16 BOMs and NUL bytes, which\n        // indicate the file was read with the wrong encoding.\n        if (\n            bytes.Length >= 2\n            && ((bytes[0] == 0xFF && bytes[1] == 0xFE) || (bytes[0] == 0xFE && bytes[1] == 0xFF))\n        )\n        {\n            throw new InvalidDataException($\"pyvenv.cfg is UTF-16 encoded; expected UTF-8/ASCII: {path}\");\n        }\n\n        var content = new UTF8Encoding(false).GetString(bytes);\n        if (content.Contains('\\0'))\n        {\n            throw new InvalidDataException($\"pyvenv.cfg contains NUL bytes; expected UTF-8/ASCII: {path}\");\n        }\n\n        return Parse(content);\n    }\n\n    /// <summary>\n    /// Gets the value of the last matching key (CPython is last-wins), or null.\n    /// Setting rewrites every matching key, appending a new key when absent.\n    /// </summary>\n    public string? this[string key]\n    {\n        get","sourceCodeStart":45,"sourceCodeEnd":81,"githubUrl":"https://github.com/LykosAI/StabilityMatrix/blob/af93d6ef57c01cd890d7e0ad0a9ea8c9fcda3002/StabilityMatrix.Core/Python/PyVenvCfg.cs#L45-L81","documentation":"PyVenvCfg.Load reads a venv's pyvenv.cfg as bytes and rejects files with a UTF-16 BOM (FF FE / FE FF), since venv config must be UTF-8/ASCII. Reading a UTF-16 file as UTF-8 would produce garbage key/value pairs, so an InvalidDataException naming the path is thrown instead.","triggerScenarios":"Calling PyVenvCfg.Load on a pyvenv.cfg that was saved as UTF-16 LE/BE — e.g. created by PowerShell 5.x ('>' redirection) or Notepad with 'Unicode' encoding.","commonSituations":"Windows tooling or scripts rewriting pyvenv.cfg with UTF-16 default encoding; a venv created/patched by a PowerShell script that out-recreated the config.","solutions":["Rewrite pyvenv.cfg as UTF-8 (no BOM) using a text editor or 'Set-Content -Encoding utf8NoBOM'","Recreate the venv so the config is generated with the correct encoding","Convert the file: read as Unicode, write back as UTF-8 before loading","Catch InvalidDataException and regenerate the config programmatically"],"exampleFix":"// before\npowershell: \"home = ...\" | Set-Content pyvenv.cfg  # UTF-16 by default\n// after\npowershell: \"home = ...\" | Set-Content -Encoding utf8NoBOM pyvenv.cfg","handlingStrategy":"validation","validationCode":"var bytes = File.ReadAllBytes(cfgPath);\nbool isUtf16 = bytes.Length >= 2 && ((bytes[0] == 0xFF && bytes[1] == 0xFE) || (bytes[0] == 0xFE && bytes[1] == 0xFF));\nif (isUtf16) { /* convert to UTF-8 before loading */ }","typeGuard":"static bool IsUtf8Clean(string path)\n{\n    var b = File.ReadAllBytes(path);\n    if (b.Length >= 2 && ((b[0] == 0xFF && b[1] == 0xFE) || (b[0] == 0xFE && b[1] == 0xFF))) return false;\n    return !new UTF8Encoding(false).GetString(b).Contains('\\0');\n}","tryCatchPattern":"try\n{\n    var cfg = PyVenvCfg.Load(cfgPath);\n}\ncatch (InvalidDataException ex)\n{\n    logger.LogWarning(ex, \"pyvenv.cfg bad encoding; regenerating\");\n    RewriteAsUtf8(cfgPath);\n    var cfg = PyVenvCfg.Load(cfgPath);\n}","preventionTips":["Never write pyvenv.cfg with PowerShell 5.x default redirection (UTF-16); use utf8NoBOM","Always save text config files as UTF-8 without BOM","Verify encoding when scripts patch venv configs"],"tags":["python","venv","encoding"],"backgroundTag":"file-encoding-mismatch","analyzedSha":"af93d6ef57c01cd890d7e0ad0a9ea8c9fcda3002","analyzedAt":"2026-09-12T19:02:43.389Z","contentChangedAt":"2026-09-12T19:02:43.389Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}