nautechsystems/nautilus_trader · error

Invalid UTF-8 in C string

Error message

Invalid UTF-8 in C string

What it means

StackStr::from_c_ptr in crates/core/src/string/stack_str.rs:206 constructs a fixed-capacity stack string from a C string pointer, converting the CStr to &str with to_str().expect("Invalid UTF-8 in C string"). It panics when the pointed-to bytes are not valid UTF-8; the doc comment also warns that violating the safety invariants (valid, null-terminated pointer) is undefined behavior, so callers must uphold those before this expect is even reached.

Source

Thrown at crates/core/src/string/stack_str.rs:206

    ///
    /// # Safety
    ///
    /// - `ptr` must be a valid, non-null pointer to a null-terminated C string.
    /// - The string must contain only valid ASCII (no interior NUL bytes).
    /// - The string must not exceed 36 characters.
    ///
    /// Violating these requirements causes a panic. If this function is called
    /// from C code, such a panic is undefined behavior.
    ///
    /// # Panics
    ///
    /// Panics if the C string contains invalid UTF-8 or violates any of the
    /// safety invariants listed above.
    #[must_use]
    pub unsafe fn from_c_ptr(ptr: *const c_char) -> Self {
        // SAFETY: Caller guarantees ptr is valid and null-terminated
        let cstr = unsafe { CStr::from_ptr(ptr) };
        let s = cstr.to_str().expect("Invalid UTF-8 in C string");
        Self::new(s)
    }

    /// Creates a [`StackStr`] from a C string pointer with validation.
    ///
    /// Returns `None` if the string is null or invalid. This is safe to call from C
    /// code for null and string-validation failures because it does not panic.
    ///
    /// # Safety
    ///
    /// - `ptr` must be null or a valid pointer to a null-terminated C string.
    #[must_use]
    pub unsafe fn from_c_ptr_checked(ptr: *const c_char) -> Option<Self> {
        if ptr.is_null() {
            return None;
        }

        // SAFETY: Caller guarantees ptr is valid and null-terminated

View on GitHub (pinned to 18893faf8b)

Solutions

1. Ensure the C producer writes UTF-8 and null-terminates the buffer (the invariants this unsafe function requires).
2. Prefer the safe validated variant (from_c_ptr's documented safe counterpart returning Option) when input trustworthiness is uncertain.
3. Truncate on a char boundary: trim the source string with floor_char_boundary-style logic before copying into the fixed buffer.
4. Re-encode non-UTF-8 sources explicitly (decode with the true encoding, encode UTF-8 with replacement) before crossing the boundary.

Example fix

// before (C producer)
strcpy(buf, "caf\xe9");              // Latin-1 bytes into shared buffer
// after
const char *s = "caf\xc3\xa9";       // UTF-8 bytes
strncpy(buf, s, buf_len - 1); buf[buf_len - 1] = '\0';

// Caller-side check before the unsafe call
let bytes = CStr::from_ptr(ptr).to_bytes();
assert!(std::str::from_utf8(bytes).is_ok(), "non-UTF-8 would panic from_c_ptr");

fn stack_str_input_ok(ptr: *const c_char) -> bool {
    if ptr.is_null() { return false; }
    unsafe { std::str::from_utf8(CStr::from_ptr(ptr).to_bytes()).is_ok() }
}

When it happens

Trigger: Calling unsafe StackStr::from_c_ptr with a pointer to non-UTF-8 bytes: legacy-encoded text, binary data mistaken for a string, or a multi-byte character truncated at the fixed capacity boundary.

Common situations: Fixed-size char buffers from C code carrying locale-encoded text; protocol frames where a length prefix was misread so the slice lands mid-character; symbol or label fields populated from non-UTF-8 vendor data.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• CStr::from_ptr failed
• C string contains invalid UTF-8
• Invalid scientific notation exponent '{exponent}': must be a
• invalid `AggressorSide` enum string value, was '{value}'
• invalid `AssetClass` enum string value, was '{value}'

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/42f0d3d3b4d7388f. Report an issue: GitHub.