{"record":{"id":"bc0af54a094ff6b1","repo":"block/buzz","slug":"io-frame-contains-invalid-utf-8","errorCode":null,"errorMessage":"io: frame contains invalid UTF-8","messagePattern":"io: frame contains invalid UTF-8","errorType":"exception","errorClass":"std::io::Error","httpStatus":null,"severity":"error","filePath":"crates/buzz-agent/src/wire.rs","lineNumber":423,"sourceCode":"            .position(|b| *b == b'\\n')\n            .map_or(chunk.len(), |i| i + 1);\n        if buf.len().saturating_add(take) > max {\n            return Err(std::io::Error::new(\n                std::io::ErrorKind::InvalidData,\n                format!(\"io: line exceeds max ({max} bytes)\"),\n            ));\n        }\n        buf.extend_from_slice(&chunk[..take]);\n        stdin.consume(take);\n        if buf.ends_with(b\"\\n\") {\n            buf.pop();\n            if buf.ends_with(b\"\\r\") {\n                buf.pop();\n            }\n            match String::from_utf8(buf) {\n                Ok(s) => return Ok(Some(s)),\n                Err(_) => {\n                    return Err(std::io::Error::new(\n                        std::io::ErrorKind::InvalidData,\n                        \"io: frame contains invalid UTF-8\",\n                    ))\n                }\n            }\n        }\n    }\n}\n\npub async fn writer_task(rx: mpsc::Receiver<WireMsg>) {\n    write_frames(rx, tokio::io::stdout()).await;\n}\n\n/// Drain `rx`, writing each frame to `out` as a newline-terminated JSON line.\n/// Generic over the sink so tests can inject an `AsyncWrite` that fails on\n/// flush; production passes stdout.\n///\n/// Both `write_all` and `flush` failure are connection-fatal: they return,","sourceCodeStart":405,"sourceCodeEnd":441,"githubUrl":"https://github.com/block/buzz/blob/eed74bde2f4797714335ac10c56c0b0244c1def4/crates/buzz-agent/src/wire.rs#L405-L441","documentation":"After a complete newline-terminated frame is read from the agent's stdin, wire::read_bounded_line converts the bytes with String::from_utf8; invalid sequences (a split multibyte character, binary garbage, non-UTF8 encoding) produce InvalidData 'io: frame contains invalid UTF-8'. NDJSON stdio framing requires every frame to be valid UTF-8 JSON, so this kills the reader loop and the agent session.","triggerScenarios":"String::from_utf8(buf) fails at crates/buzz-agent/src/wire.rs:299-311 when the frame contains: raw binary piped to the agent's stdin (e.g. `buzz-agent < image.png`), text written in a non-UTF8 encoding (latin-1 console output), a multibyte UTF-8 sequence truncated by a buggy writer, or a length-prefixed binary protocol mistakenly pointed at the NDJSON agent.","commonSituations":"Piping a file into the agent instead of a JSON frame; test harnesses writing platform-encoded strings; a producer that slices strings at byte boundaries mid-codepoint; mixing protobuf/length-prefixed framing with the line-delimited JSON protocol.","solutions":["Ensure whatever spawns buzz-agent writes UTF-8 only: in Rust use str/String (guaranteed UTF-8), in Python pass text through sys.stdout with encoding='utf-8'.","If raw bytes must be conveyed, base64-encode them inside a JSON frame rather than writing them bare to stdio.","Check the writer for byte-level slicing of strings (split at '\\n' can split a multibyte character if the writer is buggy — send whole frames only).","Capture the offending frame by logging bytes as hex (from_utf8's error reports the exact invalid offset) to identify the producer."],"exampleFix":"# before: raw binary piped to an NDJSON agent\n./buzz-agent < screenshot.png\n# io: frame contains invalid UTF-8\n\n# after: frame it as JSON\npython3 -c 'import json,base64,sys; sys.stdout.write(json.dumps({\"type\":\"file\",\"data\":base64.b64encode(open(\"screenshot.png\",\"rb\").read()).decode()})+\"\\n\")' | ./buzz-agent","handlingStrategy":"validation","validationCode":"// validate before writing to the agent's stdin\nfn is_valid_utf8_frame(bytes: &[u8]) -> bool {\n    std::str::from_utf8(bytes).is_ok()\n}","typeGuard":"fn is_utf8_str_frame(line: &str) -> bool { true } // String inputs are always UTF-8;\n// the guard matters at byte boundaries: fn is_utf8(bytes: &[u8]) -> bool { std::str::from_utf8(bytes).is_ok() }","tryCatchPattern":"// producer side: convert lossily at the boundary instead of sending invalid bytes\nlet text = String::from_utf8_lossy(&raw_bytes).into_owned();\nwriteln!(stdin, \"{text}\")?;","preventionTips":["Build frames from Rust Strings / JSON serializers — they cannot emit invalid UTF-8.","In Python/Node harnesses, force UTF-8 stdout (PYTHONIOENCODING=utf-8) when spawning the agent.","Never pipe files or binary directly into the agent; base64 inside JSON.","Split frames only on '\\n' boundaries of complete serialized messages, never mid-byte."],"tags":["rust","stdio","ndjson","utf-8","encoding","acl-protocol"],"backgroundTag":"invalid-utf8","analyzedSha":"eed74bde2f4797714335ac10c56c0b0244c1def4","analyzedAt":"2026-08-20T04:38:24.874Z","contentChangedAt":"2026-08-20T04:38:24.874Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}