block/buzz · error · std::io::Error

io: frame contains invalid UTF-8

Error message

io: frame contains invalid UTF-8

What it means

After a complete newline-terminated frame is read from the agent's stdin, wire::read_bounded_line converts the bytes with String::from_utf8; invalid sequences (a split multibyte character, binary garbage, non-UTF8 encoding) produce InvalidData 'io: frame contains invalid UTF-8'. NDJSON stdio framing requires every frame to be valid UTF-8 JSON, so this kills the reader loop and the agent session.

Source

Thrown at crates/buzz-agent/src/wire.rs:423

            .position(|b| *b == b'\n')
            .map_or(chunk.len(), |i| i + 1);
        if buf.len().saturating_add(take) > max {
            return Err(std::io::Error::new(
                std::io::ErrorKind::InvalidData,
                format!("io: line exceeds max ({max} bytes)"),
            ));
        }
        buf.extend_from_slice(&chunk[..take]);
        stdin.consume(take);
        if buf.ends_with(b"\n") {
            buf.pop();
            if buf.ends_with(b"\r") {
                buf.pop();
            }
            match String::from_utf8(buf) {
                Ok(s) => return Ok(Some(s)),
                Err(_) => {
                    return Err(std::io::Error::new(
                        std::io::ErrorKind::InvalidData,
                        "io: frame contains invalid UTF-8",
                    ))
                }
            }
        }
    }
}

pub async fn writer_task(rx: mpsc::Receiver<WireMsg>) {
    write_frames(rx, tokio::io::stdout()).await;
}

/// Drain `rx`, writing each frame to `out` as a newline-terminated JSON line.
/// Generic over the sink so tests can inject an `AsyncWrite` that fails on
/// flush; production passes stdout.
///
/// Both `write_all` and `flush` failure are connection-fatal: they return,

View on GitHub (pinned to eed74bde2f)

Solutions

1. Ensure whatever spawns buzz-agent writes UTF-8 only: in Rust use str/String (guaranteed UTF-8), in Python pass text through sys.stdout with encoding='utf-8'.
2. If raw bytes must be conveyed, base64-encode them inside a JSON frame rather than writing them bare to stdio.
3. Check the writer for byte-level slicing of strings (split at '\n' can split a multibyte character if the writer is buggy — send whole frames only).
4. Capture the offending frame by logging bytes as hex (from_utf8's error reports the exact invalid offset) to identify the producer.

Example fix

# before: raw binary piped to an NDJSON agent
./buzz-agent < screenshot.png
# io: frame contains invalid UTF-8

# after: frame it as JSON
python3 -c 'import json,base64,sys; sys.stdout.write(json.dumps({"type":"file","data":base64.b64encode(open("screenshot.png","rb").read()).decode()})+"\n")' | ./buzz-agent

// validate before writing to the agent's stdin
fn is_valid_utf8_frame(bytes: &[u8]) -> bool {
    std::str::from_utf8(bytes).is_ok()
}

fn is_utf8_str_frame(line: &str) -> bool { true } // String inputs are always UTF-8;
// the guard matters at byte boundaries: fn is_utf8(bytes: &[u8]) -> bool { std::str::from_utf8(bytes).is_ok() }

// producer side: convert lossily at the boundary instead of sending invalid bytes
let text = String::from_utf8_lossy(&raw_bytes).into_owned();
writeln!(stdin, "{text}")?;

When it happens

Trigger: String::from_utf8(buf) fails at crates/buzz-agent/src/wire.rs:299-311 when the frame contains: raw binary piped to the agent's stdin (e.g. `buzz-agent < image.png`), text written in a non-UTF8 encoding (latin-1 console output), a multibyte UTF-8 sequence truncated by a buggy writer, or a length-prefixed binary protocol mistakenly pointed at the NDJSON agent.

Common situations: Piping a file into the agent instead of a JSON frame; test harnesses writing platform-encoded strings; a producer that slices strings at byte boundaries mid-codepoint; mixing protobuf/length-prefixed framing with the line-delimited JSON protocol.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• io: line exceeds max ({max} bytes)
• failed to install rustls crypto provider
• connect to test DB
• push HTTP client
• mesh endpoint bind on {} failed: {e}

AI-assisted analysis of block/buzz@eed74bde2f (2026-08-20). Data as JSON: /api/errors/bc0af54a094ff6b1. Report an issue: GitHub.