nathanmarz/storm · critical · RuntimeException

Error when launching multilang subprocess

Error message

Error when launching multilang subprocess
${errorsString}

What it means

ShellBolt.prepare launches the external (multilang, e.g. Python) subprocess and reads its initial pid handshake. If launch() throws IOException, it is rethrown as this RuntimeException including the process's collected stderr (getErrorsString). It means the shell process failed to start or crashed before sending its pid — almost always an error printed by the child interpreter.

Solutions

Read the errorsString in the exception message — it contains the subprocess's stderr naming the real problem
Verify the script path and that it is packaged in the topology jar and executable, with a correct shebang (e.g. #!/usr/bin/env python)
SSH into a worker machine and run the script manually to reproduce missing interpreter/modules; install the needed interpreter version and packages on all supervisor nodes
Confirm the script writes its pid first thing ( storm uses ShellProcess launch handshake) — remove any code printing to stdout before the handshake

Example fix

// before (supervisor missing python deps)
RuntimeError: Error when launching multilang subprocess
Traceback: ImportError: No module named redis

// after: on each supervisor node
pip install redis
# and verify:
python /path/to/myscript.py  # runs without import errors

Defensive patterns

Strategy: validation

Validate before calling

# run on every supervisor before deploying:
test -x "$TOPOLOGY_JAR_EXTRACTED_PATH/myscript.py" && python -c "import myscript_deps" \
  || echo "script missing or interpreter/deps not installed"

Try / catch

try {
    builder.setBolt("shell", new ShellBolt(...)).shuffleGrouping(...);
    submit(topology);
} catch (RuntimeException e) {
    if (e.getMessage() != null && e.getMessage().startsWith("Error when launching multilang subprocess")) {
        LOG.error("Subprocess stderr: {}", e.getMessage()); // errorsString follows the \n
    }
    throw e;
}

Prevention

Include the multilang script in the topology jar and use a resource-relative path
Install the exact interpreter version and all pip/dependency packages on every supervisor node
Run the script standalone on a worker host before deploying
Make sure the script sends its pid first and prints nothing else to stdout before the handshake

When it happens

Trigger: The multilang script path is wrong or not executable; the interpreter (e.g. python) is missing or wrong version on the worker host; the script throws an import/parse error on startup (stderr captured in errorsString); required modules not installed in the worker environment; script exits before writing its pid.

Common situations: Python script depends on a pip package not installed on Nimbus/worker nodes; deploying to a cluster where the script path in the topology differs from the jar packaging; python2 vs python3 shebang issues; script has a syntax error or fails on import when first executed on the worker.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of nathanmarz/storm@cdb116e942 (2026-09-12). Data as JSON: /api/errors/9f625d01b549000f. Report an issue: GitHub.

Appendix: source

Thrown at storm-core/src/jvm/backtype/storm/task/ShellBolt.java:105

        _command = command;
    }

    public void prepare(Map stormConf, TopologyContext context,
                        final OutputCollector collector) {
        Object maxPending = stormConf.get(Config.TOPOLOGY_SHELLBOLT_MAX_PENDING);
        if (maxPending != null) {
           this._pendingWrites = new LinkedBlockingQueue(((Number)maxPending).intValue());
        }
        _rand = new Random();
        _process = new ShellProcess(_command);
        _collector = collector;

        try {
            //subprocesses must send their pid first thing
            Number subpid = _process.launch(stormConf, context);
            LOG.info("Launched subprocess with pid " + subpid);
        } catch (IOException e) {
            throw new RuntimeException("Error when launching multilang subprocess\n" + _process.getErrorsString(), e);
        }

        // reader
        _readerThread = new Thread(new Runnable() {
            public void run() {
                while (_running) {
                    try {
                        JSONObject action = _process.readMessage();
                        if (action == null) {
                            // ignore sync
                        }

                        String command = (String) action.get("command");
                        if(command.equals("ack")) {
                            handleAck(action);
                        } else if (command.equals("fail")) {
                            handleFail(action);
                        } else if (command.equals("error")) {

View on GitHub (pinned to cdb116e942)