PyO3/pyo3 · error

wasi strings are UTF8

Error message

wasi strings are UTF8

What it means

Converting an OsStr/Path to Python on the WASI target assumes WASI strings are valid UTF-8, so the code calls to_str().expect("wasi strings are UTF8"). If the underlying bytes are not UTF-8 (e.g. a path built from raw invalid bytes), this panics at runtime during into_pyobject.

Source

Thrown at src/conversions/std/osstr.rs:43

impl<'py> IntoPyObject<'py> for &OsStr {
    type Target = PyString;
    type Output = Bound<'py, Self::Target>;
    type Error = Infallible;

    #[cfg(feature = "experimental-inspect")]
    const OUTPUT_TYPE: PyStaticExpr = PyString::TYPE_HINT;

    fn into_pyobject(self, py: Python<'py>) -> Result<Self::Output, Self::Error> {
        // If the string is UTF-8, take the quick and easy shortcut
        #[cfg(not(target_os = "wasi"))]
        if let Some(valid_utf8_path) = self.to_str() {
            return valid_utf8_path.into_pyobject(py);
        }

        #[cfg(target_os = "wasi")]
        {
            self.to_str()
                .expect("wasi strings are UTF8")
                .into_pyobject(py)
        }

        #[cfg(any(unix, target_os = "emscripten"))]
        {
            let bytes = self.as_bytes();
            let ptr = bytes.as_ptr().cast();
            let len = bytes.len() as ffi::Py_ssize_t;
            // SAFETY: passing valid pointer to python API
            unsafe {
                // DecodeFSDefault automatically chooses an appropriate decoding mechanism to
                // parse os strings losslessly (i.e. surrogateescape most of the time)
                Ok(ffi::PyUnicode_DecodeFSDefaultAndSize(ptr, len)
                    .assume_owned(py)
                    .cast_into_unchecked())
            }
        }

View on GitHub (pinned to ac9b6899d3)

Solutions

1. Validate the OsStr/Path with to_str() before converting; handle the None case explicitly.
2. Fall back to lossy conversion via to_string_lossy() if exact bytes are not required.
3. On non-WASI targets prefer the byte-preserving branch; on WASI reject or sanitize non-UTF-8 paths at the boundary.
4. Normalize inputs early (e.g. accept String instead of PathBuf) when you control the API.

Example fix

// before
pyo3_path.to_str().expect("wasi strings are UTF8").into_pyobject(py)
// after
match pyo3_path.to_str() {
    Some(s) => s.into_pyobject(py),
    None => return Err(PyValueError::new_err("path is not valid UTF-8")),
}

// Check UTF-8 validity before converting an OsStr/Path on WASI
fn is_utf8_path(p: &std::ffi::OsStr) -> bool { p.to_str().is_some() }

fn as_utf8(p: &std::ffi::OsStr) -> Option<&str> { p.to_str() }

// Panic cannot be caught idiomatically; instead avoid expect in your own code:
let obj = match path.to_str() {
    Some(s) => s.into_pyobject(py)?,
    None => return Err(PyValueError::new_err("path is not valid UTF-8")),
};

When it happens

Trigger: Calling into_pyobject (or returning a Path/OsStr from a #[pyfunction]) on WASI where the OsStr contains non-UTF-8 bytes, e.g. OsStr::from_bytes(b"\xff") or a path read from the filesystem with invalid encoding.

Common situations: WASI builds handling paths or environment values sourced from raw bytes (host-provided preopen names, argv, or file names read via OsStrExt::from_bytes) that are not UTF-8.

Related errors

• Attaching a thread to the interpreter is prohibited while a
• Cannot attach to the Python interpreter while it is finalizi
• Attaching a thread to the interpreter is currently prohibite
• {}
• Unknown return value from PyDict_SetDefaultRef: {x}

AI-assisted analysis of PyO3/pyo3@ac9b6899d3 (2026-09-05). Data as JSON: /api/errors/aa1c463c7d2db092. Report an issue: GitHub.