{"record":{"id":"5d8a0ca18d6a1093","repo":"nathanmarz/storm","slug":"shell-process-exception-geterrorsstring","errorCode":null,"errorMessage":"Shell Process Exception:\n${getErrorsString()}","messagePattern":"Shell Process Exception:\n(.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"storm-core/src/jvm/backtype/storm/utils/ShellProcess.java","lineNumber":135,"sourceCode":"\n    private String readString() throws IOException {\n        StringBuilder line = new StringBuilder();\n\n        //synchronized (processOut) {\n            while (true) {\n                String subline = processOut.readLine();\n                if(subline==null) {\n                    StringBuilder errorMessage = new StringBuilder();\n                    errorMessage.append(\"Pipe to subprocess seems to be broken!\");\n                    if (line.length() == 0) {\n                        errorMessage.append(\" No output read.\\n\");\n                    }\n                    else {\n                        errorMessage.append(\" Currently read output: \" + line.toString() + \"\\n\");\n                    }\n                    errorMessage.append(\"Shell Process Exception:\\n\");\n                    errorMessage.append(getErrorsString() + \"\\n\");\n                    throw new RuntimeException(errorMessage.toString());\n                }\n                if(subline.equals(\"end\")) {\n                    break;\n                }\n                if(line.length()!=0) {\n                    line.append(\"\\n\");\n                }\n                line.append(subline);\n            }\n            //}\n\n        return line.toString();\n    }\n}\n","sourceCodeStart":117,"sourceCodeEnd":150,"githubUrl":"https://github.com/nathanmarz/storm/blob/cdb116e942666973bc4eaa0df098d5bab82739e7/storm-core/src/jvm/backtype/storm/utils/ShellProcess.java#L117-L150","documentation":"ShellProcess.readLineBytes throws this RuntimeException when the shell subprocess communicating over stdin/stdout with Storm breaks the protocol or dies. Before throwing, it appends the process's stderr (getErrorsString) and whatever output was already read, so the message contains the subprocess's own error output. It is Storm's way of surfacing a failed non-JVM (e.g. Python) spout/bolt process.","triggerScenarios":"readString -> readLineBytes while reading a line from a launched shell process: the process exits/dies before sending 'end', or its stdout yields non-integer length prefixes / does not match the length-prefixed protocol (e.g. reading a traceback or garbage instead of a byte count).","commonSituations":"Python or Ruby multilang bolts/spouts crashing on unhandled exceptions or missing modules; shebang interpreter not present in the worker's environment; multilang script emitting debug output to stdout that corrupts the length-prefixed protocol; wrong file path causing the subprocess to fail immediately at launch.","solutions":["Read the stderr dump appended to the exception message and fix the underlying script error (missing import, bad shebang, syntax error) it reports.","Ensure the multilang process writes only the length-prefixed protocol to stdout and sends all diagnostics to stderr (suppress print/logging to stdout).","Verify the interpreter (python/ruby/node) and script path exist on every supervisor worker machine, and that the shebang points to a valid interpreter.","Upgrade Storm or adjust ShellProcess timeout/worker settings if the subprocess is being killed by resource limits or timeouts."],"exampleFix":"// before (python bolt, pollutes protocol stdout)\nprint(\"debug info\")\n// after\nimport sys\nprint(\"debug info\", file=sys.stderr)","handlingStrategy":"try-catch","validationCode":"// before launching, verify interpreter and script exist on the worker\nassert new File(\"/usr/bin/python\").exists() || which(\"python3\") != null;\nassert new File(boltScriptPath).exists();","typeGuard":null,"tryCatchPattern":"try {\n    process.readMessage();\n} catch (RuntimeException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"Shell Process Exception\")) {\n        LOG.error(\"Multilang process failed; stderr was embedded in message\", e);\n        // treat as component failure: fail tuple / restart process\n    } else {\n        throw e;\n    }\n}","preventionTips":["Never print debug output to stdout in multilang scripts; log to stderr only.","Pin and test the interpreter version on all supervisor machines.","Wrap the multilang script's main loop in an exception handler that reports errors via the protocol instead of crashing.","Monitor worker logs for repeated Shell Process Exceptions pointing at script startup failures."],"tags":["multilang","shell-process","subprocess","storm"],"backgroundTag":"unexpected-response-shape","analyzedSha":"cdb116e942666973bc4eaa0df098d5bab82739e7","analyzedAt":"2026-09-12T14:30:00.714Z","contentChangedAt":"2026-09-12T14:30:00.714Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}