apache/hadoop · error · IOException

Fewer lines of output than expected

Error message

Fewer lines of output than expected

What it means

DF.parseOutput tokenizes the output of 'df <path>' and expects at least two non-empty lines (header + data row). If fewer than two lines were captured it throws IOException ('Fewer lines of output than expected', appending the single line if one exists). Essentially: the df subprocess produced no usable output even though the exit-code check (run later in some flows) or environment differs from what DF assumes.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DF.java:170

  @Override
  protected void parseExecResult(BufferedReader lines) throws IOException {
    output.clear();
    String line = lines.readLine();
    while (line != null) {
      output.add(line);
      line = lines.readLine();
    }
  }
  
  @VisibleForTesting
  protected void parseOutput() throws IOException {
    if (output.size() < 2) {
      StringBuilder sb = new StringBuilder("Fewer lines of output than expected");
      if (output.size() > 0) {
        sb.append(": " + output.get(0));
      }
      throw new IOException(sb.toString());
    }
    
    String line = output.get(1);
    StringTokenizer tokens =
      new StringTokenizer(line, " \t\n\r\f%");
    
    try {
      this.filesystem = tokens.nextToken();
    } catch (NoSuchElementException e) {
      throw new IOException("Unexpected empty line");
    }
    if (!tokens.hasMoreTokens()) {            // for long filesystem name
      if (output.size() > 2) {
        line = output.get(2);
      } else {
        throw new IOException("Expecting additional output after line: "
            + line);
      }

View on GitHub (pinned to 2add963021)

Solutions

1. Reproduce as the daemon user: sudo -u hdfs df -k /grid/1/dn - if that fails, fix the OS layer (install coreutils, adjust PATH in the service unit).
2. For containers, use an image that ships standard coreutils and does not seccomp-block exec of /bin/df.
3. Upgrade Hadoop - later DF versions harden output handling; if stuck, point data dirs at a filesystem whose df behaves standardly.
4. Check the daemon's environment (systemd unit Environment=, container entrypoint) for PATH without /usr/bin or /bin.

Example fix

# before: DataNode in distroless-like container fails DF.getMount()
# IOException: Fewer lines of output than expected

# after: ensure coreutils df exists for the daemon user
RUN apt-get install -y coreutils   # Dockerfile
# verify: sudo -u hdfs df -k /grid/1/dn  ->  2+ lines of output

// smoke-test df availability before relying on DF
Process p = new ProcessBuilder("df", "-k", dir).start();
int lines = countNonEmptyLines(p.getInputStream());
if (lines < 2) throw new IOException("df unusable for " + dir + " - fix PATH/coreutils");

try {
  long avail = df.getAvailable();
} catch (IOException e) { // 'Fewer lines of output than expected'
  // degrade to java.io.File.getUsableSpace() for capacity checks
}

When it happens

Trigger: new DF(dir).getMount()/available() running 'df -k <dir>' where stdout is empty or one blank line: df binary not on PATH in a minimal container, df blocked by sandbox/seccomp policies, df writing errors only to stderr, or the process output being swallowed by locale/encoding issues.

Common situations: Hadoop daemons in slim Docker images (no coreutils), chroot'd or gVisor/Kata sandboxes filtering exec, PATH stripped by systemd unit or cron-launched daemons, or an exotic OS where df prints differently.

Related errors

• Unexpected empty line
• Expecting additional output after line: ${line}
• Could not parse line: ${line}
• Specified path ${path}does not exist
• Invalid UID, could not determine effective user

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/479b8ae4817f2b5a. Report an issue: GitHub.