{"record":{"id":"479b8ae4817f2b5a","repo":"apache/hadoop","slug":"fewer-lines-of-output-than-expected","errorCode":null,"errorMessage":"Fewer lines of output than expected","messagePattern":"Fewer lines of output than expected","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DF.java","lineNumber":170,"sourceCode":"\n  @Override\n  protected void parseExecResult(BufferedReader lines) throws IOException {\n    output.clear();\n    String line = lines.readLine();\n    while (line != null) {\n      output.add(line);\n      line = lines.readLine();\n    }\n  }\n  \n  @VisibleForTesting\n  protected void parseOutput() throws IOException {\n    if (output.size() < 2) {\n      StringBuilder sb = new StringBuilder(\"Fewer lines of output than expected\");\n      if (output.size() > 0) {\n        sb.append(\": \" + output.get(0));\n      }\n      throw new IOException(sb.toString());\n    }\n    \n    String line = output.get(1);\n    StringTokenizer tokens =\n      new StringTokenizer(line, \" \\t\\n\\r\\f%\");\n    \n    try {\n      this.filesystem = tokens.nextToken();\n    } catch (NoSuchElementException e) {\n      throw new IOException(\"Unexpected empty line\");\n    }\n    if (!tokens.hasMoreTokens()) {            // for long filesystem name\n      if (output.size() > 2) {\n        line = output.get(2);\n      } else {\n        throw new IOException(\"Expecting additional output after line: \"\n            + line);\n      }","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DF.java#L152-L188","documentation":"DF.parseOutput tokenizes the output of 'df <path>' and expects at least two non-empty lines (header + data row). If fewer than two lines were captured it throws IOException ('Fewer lines of output than expected', appending the single line if one exists). Essentially: the df subprocess produced no usable output even though the exit-code check (run later in some flows) or environment differs from what DF assumes.","triggerScenarios":"new DF(dir).getMount()/available() running 'df -k <dir>' where stdout is empty or one blank line: df binary not on PATH in a minimal container, df blocked by sandbox/seccomp policies, df writing errors only to stderr, or the process output being swallowed by locale/encoding issues.","commonSituations":"Hadoop daemons in slim Docker images (no coreutils), chroot'd or gVisor/Kata sandboxes filtering exec, PATH stripped by systemd unit or cron-launched daemons, or an exotic OS where df prints differently.","solutions":["Reproduce as the daemon user: sudo -u hdfs df -k /grid/1/dn - if that fails, fix the OS layer (install coreutils, adjust PATH in the service unit).","For containers, use an image that ships standard coreutils and does not seccomp-block exec of /bin/df.","Upgrade Hadoop - later DF versions harden output handling; if stuck, point data dirs at a filesystem whose df behaves standardly.","Check the daemon's environment (systemd unit Environment=, container entrypoint) for PATH without /usr/bin or /bin."],"exampleFix":"# before: DataNode in distroless-like container fails DF.getMount()\n# IOException: Fewer lines of output than expected\n\n# after: ensure coreutils df exists for the daemon user\nRUN apt-get install -y coreutils   # Dockerfile\n# verify: sudo -u hdfs df -k /grid/1/dn  ->  2+ lines of output","handlingStrategy":"try-catch","validationCode":"// smoke-test df availability before relying on DF\nProcess p = new ProcessBuilder(\"df\", \"-k\", dir).start();\nint lines = countNonEmptyLines(p.getInputStream());\nif (lines < 2) throw new IOException(\"df unusable for \" + dir + \" - fix PATH/coreutils\");","typeGuard":null,"tryCatchPattern":"try {\n  long avail = df.getAvailable();\n} catch (IOException e) { // 'Fewer lines of output than expected'\n  // degrade to java.io.File.getUsableSpace() for capacity checks\n}","preventionTips":["Ship coreutils in daemon container images and keep /usr/bin on PATH in systemd units.","Run 'sudo -u <daemon> df -k <dir>' as part of node bring-up validation.","Sandbox policies (seccomp/gVisor) must allow exec of /bin/df for DF-based checks."],"tags":["hadoop","disk","df","system-command","environment","container"],"backgroundTag":"df-command-output-parse-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}