{"record":{"id":"229f9a4cee75fd30","repo":"rtk-ai/rtk","slug":"failed-to-read-stdin","errorCode":null,"errorMessage":"Failed to read stdin: {}","messagePattern":"Failed to read stdin: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/cmds/system/pipe_cmd.rs","lineNumber":257,"sourceCode":"    std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| filter_fn(input)))\n        .unwrap_or_else(|_| {\n            eprintln!(\"[rtk] warning: filter panicked — passing through raw output\");\n            input.to_string()\n        })\n}\n\npub fn run(filter_name: Option<&str>, passthrough: bool) -> Result<()> {\n    if passthrough {\n        std::io::copy(&mut std::io::stdin(), &mut std::io::stdout())\n            .map_err(|e| anyhow::anyhow!(\"Failed to relay stdin: {}\", e))?;\n        return Ok(());\n    }\n\n    let mut buf = String::new();\n    std::io::stdin()\n        .take((RAW_CAP + 1) as u64)\n        .read_to_string(&mut buf)\n        .map_err(|e| anyhow::anyhow!(\"Failed to read stdin: {}\", e))?;\n    if buf.len() > RAW_CAP {\n        anyhow::bail!(\"stdin exceeds {} byte limit\", RAW_CAP);\n    }\n\n    let filter_fn = match filter_name {\n        Some(name) => resolve_filter(name).ok_or_else(|| {\n            anyhow::anyhow!(\n                \"Unknown filter '{}'. Available: cargo-test, pytest, go-test, go-build, \\\n                 tsc, vitest, grep, rg, find, fd, git-log, git-diff, git-status, \\\n                 log, mypy, ruff-check, ruff-format, prettier, phpunit, pest, \\\n                 paratest, php-test, ecs, phpstan, pint\",\n                name\n            )\n        })?,\n        None => auto_detect_filter(&buf),\n    };\n\n    let output = apply_filter(filter_fn, &buf);","sourceCodeStart":239,"sourceCodeEnd":275,"githubUrl":"https://github.com/rtk-ai/rtk/blob/d977e1c31621fe8704e6500ceeb9c7a0de2b6836/src/cmds/system/pipe_cmd.rs#L239-L275","documentation":"Filtered pipe mode reads ALL of stdin into a String via read_to_string, which requires the stream to be valid UTF-8; any IO error or invalid byte sequence is reported as `Failed to read stdin: {e}`. The read is capped at RAW_CAP+1 bytes (10 MiB) so oversize input fails on the explicit size check, not here — this error is about readability/IO, not size.","triggerScenarios":"Piping non-UTF-8 bytes into `rtk pipe` or `rtk pipe --filter <name>`: `gzip -dc app.log.gz | rtk pipe` (compressed bytes), tar streams, images/binaries, latin-1 or UTF-16 logs; also a stdin fd that returns an error (closed or bad descriptor).","commonSituations":"Forgetting that .gz/.zst output is binary until decompressed; Windows-origin logs in UTF-16; locale-mojibake build logs with stray 0x80-0xFF bytes; piping `cat` of a .node/.wasm asset.","solutions":["Switch to byte-safe passthrough: `gzip -dc app.log.gz | rtk pipe --passthrough` (io::copy, no UTF-8 requirement, no cap)","Convert to text before filtering: `gzip -dc app.log.gz | strings | rtk pipe`, or `iconv -f latin1 -t utf8 logs.txt | rtk pipe`","Pre-verify the stream: `iconv -f utf-8 -t utf-8 < input > /dev/null` exits non-zero on invalid UTF-8"],"exampleFix":"# before: binary stream -> \"Failed to read stdin: stream did not contain valid UTF-8\"\ngzip -dc app.log.gz | rtk pipe\n\n# after\ngzip -dc app.log.gz | rtk pipe --passthrough   # byte-for-byte, no UTF-8 need\ngzip -dc app.log.gz | zcat -f | strings | rtk pipe  # or make it textual first","handlingStrategy":"validation","validationCode":"bash:\n# verify the stream is UTF-8 before handing it to rtk pipe\niconv -f utf-8 -t utf-8 < input > /dev/null 2>&1 || { echo \"not UTF-8 — use rtk pipe --passthrough\" >&2; exit 2; }\nrtk pipe --filter cargo-test < input","typeGuard":"rust:\nfn stdin_is_utf8(buf: &[u8]) -> bool {\n    std::str::from_utf8(buf).is_ok()\n}","tryCatchPattern":"rust:\nmatch pipe_cmd::run(filter_name, false) {\n    Err(e) if e.to_string().contains(\"stream did not contain valid UTF-8\")\n           || e.to_string().contains(\"Failed to read stdin\") => {\n        // binary input: redo as byte-safe passthrough, no UTF-8 requirement\n        pipe_cmd::run(None, true)\n    }\n    r => r,\n}","preventionTips":["Decompress before filtering: pipe gunzipped text, never raw .gz bytes","Route binary or unknown-encoding streams to `rtk pipe --passthrough`","Normalize legacy encodings (latin-1, UTF-16) with iconv upstream","Remember the cap check happens after the UTF-8 read — fix encoding first when debugging"],"tags":["stdin","utf-8","io","pipe","encoding"],"backgroundTag":null,"analyzedSha":"d977e1c31621fe8704e6500ceeb9c7a0de2b6836","analyzedAt":"2026-08-16T05:40:46.291Z","schemaVersion":2},"datasetVersion":"2026-08-16T08:17:34.114Z"}