nathanmarz/storm · error · RuntimeException

Error when launching multilang subprocess

Error message

Error when launching multilang subprocess
${_process.getErrorsString()}

What it means

ShellSpout launches a language-agnostic subprocess (ShellProcess, typically a Python 'multilang' bolt/spout script) when the spout is opened. If launching that subprocess fails with an IOException, the spout wraps it in a RuntimeException('Error when launching multilang subprocess\n' + _process.getErrorsString(), e), including whatever stderr output the process produced, so the topology fails fast with diagnostic detail.

Solutions

Read the stack trace below this message: _process.getErrorsString() contains the subprocess's stderr (e.g. 'ModuleNotFoundError', 'python: command not found') and fix that root cause first.
Verify the interpreter and script exist and are executable on every worker node (e.g. 'which python', check the multilang resources are in the submitted jar).
Install required Python dependencies on all workers (or vendor them in the jar/virtualenv shipped with the topology).
Test the spout script locally with the same command ('python spout.py') to reproduce import/path errors quickly.
Ensure worker's configured childenv/path includes the interpreter (set 'supervisor.worker...childopts' or correct PATH) if the interpreter lives in a nonstandard location.

Defensive patterns

Strategy: try-catch

Validate before calling

// Before submitting the topology, verify interpreter and script are reachable:
Process p = new ProcessBuilder("python", "--version").redirectErrorStream(true).start();
if (p.waitFor() != 0) throw new IllegalStateException("python interpreter missing on worker");
if (!new File("multilang/resources/spout.py").canExecute()) {
    throw new IllegalStateException("multilang spout script missing from topology resources");
}

Try / catch

try {
    Number pid = process.launch(stormConf, context);
} catch (IOException e) {
    LOG.error("multilang launch failed:\n" + process.getErrorsString(), e);
    throw new RuntimeException("Cannot start multilang spout; check stderr above", e);
}

Prevention

Install the interpreter and all multilang dependencies on every supervisor node, matching the version used in dev
Package multilang resources inside the topology jar and verify with 'jar tf' before submission
Run the subprocess command manually on a worker host as the storm user to surface PATH/permission issues early
Check the stderr captured in getErrorsString() — it names the exact missing module or file
Pin the interpreter via absolute path in the ShellSpout command if PATH differs across nodes

When it happens

Trigger: ShellSpout.open() calls _process.launch(stormConf, context) which spawns the configured command (e.g. 'python spout.py'); the process executable or script is missing/not executable, the interpreter fails immediately (import error, syntax error, missing pip package), the working directory is wrong on the worker, or the script writes errors and exits before completing the handshake.

Common situations: Deploying a topology with a Python spout to workers lacking the script's dependencies (no pip module installed, python version mismatch); the multilang resources dir not included in the topology jar; the command line in ShellSpout's constructor ('python', 'spout.py') not present on worker nodes; file permissions preventing execution.

Related errors

AI-assisted analysis of nathanmarz/storm@cdb116e942 (2026-09-12). Data as JSON: /api/errors/f16a9cc4b82b1b96. Report an issue: GitHub.

Appendix: source

Thrown at storm-core/src/jvm/backtype/storm/spout/ShellSpout.java:56

    public ShellSpout(ShellComponent component) {
        this(component.get_execution_command(), component.get_script());
    }
    
    public ShellSpout(String... command) {
        _command = command;
    }
    
    public void open(Map stormConf, TopologyContext context,
                     SpoutOutputCollector collector) {
        _process = new ShellProcess(_command);
        _collector = collector;

        try {
            Number subpid = _process.launch(stormConf, context);
            LOG.info("Launched subprocess with pid " + subpid);
        } catch (IOException e) {
            throw new RuntimeException("Error when launching multilang subprocess\n" + _process.getErrorsString(), e);
        }
    }

    public void close() {
        _process.destroy();
    }

    private JSONObject _next;
    public void nextTuple() {
        if (_next == null) {
            _next = new JSONObject();
            _next.put("command", "next");
        }

        querySubprocess(_next);
    }

    private JSONObject _ack;

View on GitHub (pinned to cdb116e942)