What causes "stdin exceeds {} byte limit" in rtk-ai/rtk?

Trim upstream: `tail -c 10485760 huge.log | rtk pipe` or pre-filter: `grep -E 'FAIL|^error' huge.log | rtk pipe`

stdin exceeds {} byte limit

Error message

stdin exceeds {} byte limit

What it means

Filtered pipe mode caps captured stdin at RAW_CAP = 10_485_760 bytes (10 MiB, src/core/stream.rs:271). The reader takes RAW_CAP+1 bytes so oversize input is detected and rejected with a hard error rather than silently truncating a filter mid-line. Passthrough mode (`--passthrough`) uses io::copy and has no cap.

Source

Thrown at src/cmds/system/pipe_cmd.rs:259

            eprintln!("[rtk] warning: filter panicked — passing through raw output");
            input.to_string()
        })
}

pub fn run(filter_name: Option<&str>, passthrough: bool) -> Result<()> {
    if passthrough {
        std::io::copy(&mut std::io::stdin(), &mut std::io::stdout())
            .map_err(|e| anyhow::anyhow!("Failed to relay stdin: {}", e))?;
        return Ok(());
    }

    let mut buf = String::new();
    std::io::stdin()
        .take((RAW_CAP + 1) as u64)
        .read_to_string(&mut buf)
        .map_err(|e| anyhow::anyhow!("Failed to read stdin: {}", e))?;
    if buf.len() > RAW_CAP {
        anyhow::bail!("stdin exceeds {} byte limit", RAW_CAP);
    }

    let filter_fn = match filter_name {
        Some(name) => resolve_filter(name).ok_or_else(|| {
            anyhow::anyhow!(
                "Unknown filter '{}'. Available: cargo-test, pytest, go-test, go-build, \
                 tsc, vitest, grep, rg, find, fd, git-log, git-diff, git-status, \
                 log, mypy, ruff-check, ruff-format, prettier, phpunit, pest, \
                 paratest, php-test, ecs, phpstan, pint",
                name
            )
        })?,
        None => auto_detect_filter(&buf),
    };

    let output = apply_filter(filter_fn, &buf);
    let shown = never_worse(&buf, &output);
    print!("{}", shown);

View on GitHub (pinned to d977e1c316)

Solutions

Trim upstream: `tail -c 10485760 huge.log | rtk pipe` or pre-filter: `grep -E 'FAIL|^error' huge.log | rtk pipe`
Use uncapped passthrough when you need everything: `cat huge.log | rtk pipe --passthrough`
Reduce verbosity at the source (test reporters, log levels) or split the stream per module/file

Example fix

# before: 40 MiB log -> "stdin exceeds 10485760 byte limit"
cat all-tests.log | rtk pipe

# after
tail -c 10485760 all-tests.log | rtk pipe      # last 10 MiB
cat all-tests.log | rtk pipe --passthrough     # no cap, unfiltered
grep -E 'FAILED|^error' all-tests.log | rtk pipe

Defensive patterns

Strategy: validation

Validate before calling

bash:
CAP=10485760  # RAW_CAP = 10 MiB (src/core/stream.rs)
SIZE=$(wc -c < input.log)
if [ "$SIZE" -gt "$CAP" ]; then
  tail -c "$CAP" input.log | rtk pipe      # or: rtk pipe --passthrough < input.log
else
  rtk pipe < input.log
fi

Try / catch

bash:
rtk pipe < input.log 2>err.txt || {
  grep -q 'stdin exceeds' err.txt && { tail -c 10485760 input.log | rtk pipe; }
} || rtk pipe --passthrough < input.log

Prevention

Know the cap: 10 MiB (10_485_760 bytes) for filtered pipe mode; passthrough has none
Pre-trim logs with tail/grep so rtk sees the useful tail, not the whole history
Split monorepo logs per crate/module before filtering
Lower upstream verbosity (reporter modes, --quiet) instead of shipping raw dumps

When it happens

Trigger: Feeding more than 10 MiB into `rtk pipe` or `rtk pipe --filter cargo-test`: `cat monorepo-test.log | rtk pipe` with a 40 MiB log; `rtk git diff | ...` chains that accumulate; verbose CI/e2e dumps. The error fires exactly when buf.len() > 10485760.

Common situations: Monorepo test/build logs with debug verbosity; Playwright/Cypress full traces; `git log -p` over long histories piped raw; combining several logs with cat before rtk pipe.

Related errors

AI-assisted analysis of rtk-ai/rtk@d977e1c316 (2026-08-16). Data as JSON: /api/errors/9b4328f4c9394101. Report an issue: GitHub.