{"record":{"id":"45061b63586d6959","repo":"wavetermdev/waveterm","slug":"file-s-contains-binary-data-and-cannot-be-uploade","errorCode":null,"errorMessage":"file %s contains binary data and cannot be uploaded as text","messagePattern":"file (.+?) contains binary data and cannot be uploaded as text","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/wsh/cmd/wshcmd-ai.go","lineNumber":146,"sourceCode":"\t\t\t\tmimeType = \"directory\"\n\t\t\t} else {\n\t\t\t\tdata, err = os.ReadFile(filePath)\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn fmt.Errorf(\"reading file %s: %w\", filePath, err)\n\t\t\t\t}\n\t\t\t\tfileName = absPath\n\t\t\t\tmimeType = detectMimeType(data)\n\t\t\t}\n\t\t}\n\n\t\tisPDF := mimeType == \"application/pdf\"\n\t\tisImage := strings.HasPrefix(mimeType, \"image/\")\n\t\tisDirectory := mimeType == \"directory\"\n\n\t\tif !isPDF && !isImage && !isDirectory {\n\t\t\tmimeType = \"text/plain\"\n\t\t\tif utilfn.ContainsBinaryData(data) {\n\t\t\t\treturn fmt.Errorf(\"file %s contains binary data and cannot be uploaded as text\", fileName)\n\t\t\t}\n\t\t}\n\n\t\tmaxSize, sizeStr := getMaxFileSize(mimeType)\n\t\tif len(data) > maxSize {\n\t\t\treturn fmt.Errorf(\"file %s exceeds maximum size of %s for %s files\", fileName, sizeStr, mimeType)\n\t\t}\n\n\t\tallFiles = append(allFiles, wshrpc.AIAttachedFile{\n\t\t\tName:   fileName,\n\t\t\tType:   mimeType,\n\t\t\tSize:   len(data),\n\t\t\tData64: base64.StdEncoding.EncodeToString(data),\n\t\t})\n\t}\n\n\ttabId := os.Getenv(\"WAVETERM_TABID\")\n\tif tabId == \"\" {","sourceCodeStart":128,"sourceCodeEnd":164,"githubUrl":"https://github.com/wavetermdev/waveterm/blob/a4447c1563b2df285ab89e76c82f91e1a1a49c1e/cmd/wsh/cmd/wshcmd-ai.go#L128-L164","documentation":"After detecting the MIME type, `wsh ai` treats non-PDF, non-image, non-directory attachments as plain text and runs utilfn.ContainsBinaryData on the bytes. If binary content is found (NUL bytes / non-UTF8 sequences), the upload is rejected because the AI text attachment pipeline cannot represent it. fileName is the absolute path (or \"stdin\").","triggerScenarios":"Attaching any binary file that http.DetectContentType doesn't classify as image or PDF — executables, zip/tar archives, sqlite db files, .docx, pickled files, or even a text file with a stray NUL byte or invalid UTF-8; also piping binary data via `binaryproc | wsh ai -`.","commonSituations":"`wsh ai app.zip` or `wsh ai data.db`; attaching documents saved in binary formats (.docx, .xlsx); corrupted \"text\" files with encoding issues; accidental binary output piped to stdin.","solutions":["Convert the file to text first: unzip the archive and attach members, export the document as .txt/.md/.csv, or dump the db as SQL/text.","If it's an image or PDF, verify its extension/content so DetectContentType classifies it correctly (a renamed image may still detect correctly; check the wrapped detection).","For encodings, convert to UTF-8: `iconv -f <enc> -t utf-8 file > file.utf8` then attach the converted file.","Describe or excerpt the binary content via `--message` or `strings file | wsh ai -` if only structure matters.","For a text file falsely flagged, find the offending bytes: `grep -P '[\\x00]' file` and clean them (e.g. `tr -d '\\000'`)"],"exampleFix":"// before\n$ wsh ai results.db\n// error: file /path/results.db contains binary data...\n// after\n$ sqlite3 results.db .dump > results.sql\n$ wsh ai results.sql","handlingStrategy":"validation","validationCode":"is_text() { LC_ALL=C grep -qP '[\\x00]' \"$1\" 2>/dev/null && return 1 || return 0; }\nfor f in \"$@\"; do\n  [ \"$f\" = \"-\" ] && continue\n  case \"$f\" in *.pdf|*.png|*.jpg|*.jpeg|*.gif|*.webp) continue ;; esac\n  if ! is_text \"$f\"; then\n    echo \"binary file, convert or extract text first: $f\" >&2\n    exit 1\n  fi\ndone","typeGuard":"func isTextFile(path string) bool {\n    data, err := os.ReadFile(path)\n    if err != nil {\n        return false\n    }\n    mime := strings.Split(http.DetectContentType(data), \";\")[0]\n    if mime == \"application/pdf\" || strings.HasPrefix(mime, \"image/\") {\n        return true\n    }\n    return !utilfn.ContainsBinaryData(data)\n}","tryCatchPattern":"if err := runAI(args); err != nil {\n    if strings.Contains(err.Error(), \"contains binary data\") {\n        return fmt.Errorf(\"convert the file to text (export/unzip/UTF-8) before attaching\")\n    }\n    return err\n}","preventionTips":["Only attach text formats (.txt/.md/.json/.csv/source code); export .docx/.xlsx to text first.","Extract archive members before attaching; never attach .zip/.tar/.db directly.","Sanitize suspect text files: strip NUL bytes (`tr -d '\\000'`) and re-encode with iconv to UTF-8.","For binary inputs, pre-convert to text (e.g. `sqlite3 db .dump`, `strings file`)."],"tags":["binary-data","file-format","validation","wsh"],"backgroundTag":"binary-file-not-allowed-as-text","analyzedSha":"a4447c1563b2df285ab89e76c82f91e1a1a49c1e","analyzedAt":"2026-09-01T15:26:23.972Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}