{"record":{"id":"ca440812b259277b","repo":"Hmbown/CodeWhale","slug":"file-appears-to-be-binary","errorCode":null,"errorMessage":"file appears to be binary","messagePattern":"file appears to be binary","errorType":"exception","errorClass":"std::io::Error","httpStatus":null,"severity":"warning","filePath":"crates/tui/src/tui/file_mention.rs","lineNumber":1400,"sourceCode":"        .take(MAX_MENTION_FILE_BYTES + 1)\n        .read_to_end(&mut buffer)?;\n    let truncated = buffer.len() as u64 > MAX_MENTION_FILE_BYTES;\n    if truncated {\n        buffer.truncate(MAX_MENTION_FILE_BYTES as usize);\n        // Round down to the nearest valid UTF-8 character boundary so a\n        // multi-byte sequence (CJK, emoji, etc.) is never split at the cut point.\n        // Only adjust when error_len() is None — that means truncation landed\n        // mid-sequence (incomplete tail).  A Some(_) error_len means the file\n        // genuinely contains invalid UTF-8 bytes; leave the buffer intact so\n        // the from_utf8 call below returns the correct \"file is not UTF-8\" error.\n        if let Err(e) = std::str::from_utf8(&buffer)\n            && e.error_len().is_none()\n        {\n            buffer.truncate(e.valid_up_to());\n        }\n    }\n    if buffer.contains(&0) {\n        return Err(std::io::Error::new(\n            std::io::ErrorKind::InvalidData,\n            \"file appears to be binary\",\n        ));\n    }\n    let text = std::str::from_utf8(&buffer)\n        .map_err(|_| std::io::Error::new(std::io::ErrorKind::InvalidData, \"file is not UTF-8\"))?\n        .to_string();\n    Ok((text, truncated))\n}\n\nfn is_media_path(path: &Path) -> bool {\n    let Some(ext) = path.extension().and_then(|ext| ext.to_str()) else {\n        return false;\n    };\n    matches!(\n        ext.to_ascii_lowercase().as_str(),\n        \"png\"\n            | \"jpg\"","sourceCodeStart":1382,"sourceCodeEnd":1418,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/0c42157ee52f9d55af2b506d71b46249910f77d3/crates/tui/src/tui/file_mention.rs#L1382-L1418","documentation":"InvalidData ('file appears to be binary') raised by the @-mention file reader in the TUI when the read prefix buffer contains a NUL byte (0x00). NUL is the classic binary-file marker: the check runs after a partial-read truncation fix-up, so a cut mid-multibyte-sequence is repaired first and only genuine NUL content triggers this. Media files (png/jpg/...) are detected earlier by extension via is_media_path and never reach this check.","triggerScenarios":"Using @-mention completion on a file whose first N bytes contain a NUL byte: executables, images with wrong/unknown extension, database files, serialized blobs, or UTF-16 text (whose ASCII bytes interleave with 0x00).","commonSituations":"Mentioning a data/image file that lacks a recognized media extension (e.g. .bin, .dat, .pickle); UTF-16 exports from Windows tools; attempting to pull a binary into the prompt as context.","solutions":["Do not @-mention the binary file; reference its path as plain text so the model can use a byte-capable tool instead","If you believe it is text, check for UTF-16: `file -i path` — convert with `iconv -f UTF-16 -t UTF-8`","Rename actual media files to their real extension so is_media_path routes them correctly"],"exampleFix":"# before\n$ file -i export.dat\nexport.dat: application/octet-stream  # @-mention -> InvalidData: file appears to be binary\n\n# after\n$ iconv -f UTF-16LE -t UTF-8 export.dat > export.txt   # if it was UTF-16 text\n# then @-mention export.txt; otherwise just type the path instead of mentioning it","handlingStrategy":"validation","validationCode":"// Cheap pre-check for @-mention candidates: NUL sniff on a prefix.\nlet mut probe = vec![0u8; 4096];\nlet n = std::fs::File::open(p)?.read(&mut probe)?;\nif probe[..n].contains(&0) {\n    return Ok(mention_as_path_only(p)); // do not attempt text ingestion\n}","typeGuard":"fn is_binary_file_error(e: &std::io::Error) -> bool {\n    e.kind() == std::io::ErrorKind::InvalidData && e.to_string().contains(\"appears to be binary\")\n}","tryCatchPattern":null,"preventionTips":["Give media files their real extensions so they route through media handling, not the text reader","Mention binaries by path text, not by @-mention ingestion","Remember UTF-16 files contain NULs and read as 'binary' — convert them to UTF-8"],"tags":["file-mention","binary-detection","tui","utf-16","rust"],"backgroundTag":"binary-file-detection","analyzedSha":"0c42157ee52f9d55af2b506d71b46249910f77d3","analyzedAt":"2026-08-20T21:50:45.477Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}