nathanmarz/storm · error · RuntimeException

Shell Process Exception

Error message

Shell Process Exception:
${getErrorsString()}

What it means

ShellProcess.readLineBytes throws this RuntimeException when the shell subprocess communicating over stdin/stdout with Storm breaks the protocol or dies. Before throwing, it appends the process's stderr (getErrorsString) and whatever output was already read, so the message contains the subprocess's own error output. It is Storm's way of surfacing a failed non-JVM (e.g. Python) spout/bolt process.

Solutions

1. Read the stderr dump appended to the exception message and fix the underlying script error (missing import, bad shebang, syntax error) it reports.
2. Ensure the multilang process writes only the length-prefixed protocol to stdout and sends all diagnostics to stderr (suppress print/logging to stdout).
3. Verify the interpreter (python/ruby/node) and script path exist on every supervisor worker machine, and that the shebang points to a valid interpreter.
4. Upgrade Storm or adjust ShellProcess timeout/worker settings if the subprocess is being killed by resource limits or timeouts.

Example fix

// before (python bolt, pollutes protocol stdout)
print("debug info")
// after
import sys
print("debug info", file=sys.stderr)

// before launching, verify interpreter and script exist on the worker
assert new File("/usr/bin/python").exists() || which("python3") != null;
assert new File(boltScriptPath).exists();

try {
    process.readMessage();
} catch (RuntimeException e) {
    if (e.getMessage() != null && e.getMessage().contains("Shell Process Exception")) {
        LOG.error("Multilang process failed; stderr was embedded in message", e);
        // treat as component failure: fail tuple / restart process
    } else {
        throw e;
    }
}

When it happens

Trigger: readString -> readLineBytes while reading a line from a launched shell process: the process exits/dies before sending 'end', or its stdout yields non-integer length prefixes / does not match the length-prefixed protocol (e.g. reading a traceback or garbage instead of a byte count).

Common situations: Python or Ruby multilang bolts/spouts crashing on unhandled exceptions or missing modules; shebang interpreter not present in the worker's environment; multilang script emitting debug output to stdout that corrupts the length-prefixed protocol; wrong file path causing the subprocess to fail immediately at launch.

Related errors

• Error when launching multilang subprocess
• Error when launching multilang subprocess
• Error during multilang processing
• Acked a non-existent or already acked/failed id
• Failed a non-existent or already acked/failed id

AI-assisted analysis of nathanmarz/storm@cdb116e942 (2026-09-12). Data as JSON: /api/errors/5d8a0ca18d6a1093. Report an issue: GitHub.

Appendix: source

Thrown at storm-core/src/jvm/backtype/storm/utils/ShellProcess.java:135

    private String readString() throws IOException {
        StringBuilder line = new StringBuilder();

        //synchronized (processOut) {
            while (true) {
                String subline = processOut.readLine();
                if(subline==null) {
                    StringBuilder errorMessage = new StringBuilder();
                    errorMessage.append("Pipe to subprocess seems to be broken!");
                    if (line.length() == 0) {
                        errorMessage.append(" No output read.\n");
                    }
                    else {
                        errorMessage.append(" Currently read output: " + line.toString() + "\n");
                    }
                    errorMessage.append("Shell Process Exception:\n");
                    errorMessage.append(getErrorsString() + "\n");
                    throw new RuntimeException(errorMessage.toString());
                }
                if(subline.equals("end")) {
                    break;
                }
                if(line.length()!=0) {
                    line.append("\n");
                }
                line.append(subline);
            }
            //}

        return line.toString();
    }
}

View on GitHub (pinned to cdb116e942)