apache/hadoop · error · IOException

Could not parse line: ${line}

Error message

Could not parse line: ${line}

What it means

After locating the numeric row, DF parses five fields in order (capacity, used, available, pct, mount) via StringTokenizer; if a token is missing (NoSuchElementException) it rethrows IOException 'Could not parse line'. The df line simply had fewer columns than the expected layout - usually a percentage or mount-point field that this OS/filesystem combination does not emit.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DF.java:199

    }
    if (!tokens.hasMoreTokens()) {            // for long filesystem name
      if (output.size() > 2) {
        line = output.get(2);
      } else {
        throw new IOException("Expecting additional output after line: "
            + line);
      }
      tokens = new StringTokenizer(line, " \t\n\r\f%");
    }

    try {
      Long.parseLong(tokens.nextToken()); // capacity
      Long.parseLong(tokens.nextToken()); // used
      Long.parseLong(tokens.nextToken()); // available
      Integer.parseInt(tokens.nextToken()); // pct used
      this.mount = tokens.nextToken();
    } catch (NoSuchElementException e) {
      throw new IOException("Could not parse line: " + line);
    } catch (NumberFormatException e) {
      throw new IOException("Could not parse line: " + line);
    }
  }

  private void verifyExitCode() throws IOException {
    if (getExitCode() != 0) {
      StringBuilder sb =
          new StringBuilder("df could not be run successfully: ");
      for (String line : output) {
        sb.append(line);
      }
      throw new IOException(sb.toString());
    }
  }

  public static void main(String[] args) throws Exception {
    String path = ".";

View on GitHub (pinned to 2add963021)

Solutions

1. Run sudo -u <daemon> df -k <datadir> and count columns against 'Filesystem 1K-blocks Used Available Use% Mounted on'; if a column is missing, that mount is unusable for Hadoop data dirs on this version.
2. Move data dirs to a standard local filesystem (ext4/xfs) and keep FUSE/exotic mounts out of dfs.datanode.data.dir.
3. Upgrade Hadoop for more tolerant DF parsing, or avoid DF-based checks on such volumes in your own code.

Example fix

# before
# sshfs mount: df -k /mnt/remote prints a row with no mount-point column
# DF.parseOutput(): IOException: Could not parse line: sshfs# ...

# after
# datanode uses a real local volume instead
# hdfs-site.xml: dfs.datanode.data.dir = /grid1/dn   (ext4/xfs, full df row)

// pre-flight: data row must have >= 6 columns matching
// 'Filesystem 1K-blocks Used Available Use% Mounted on'
Process p = new ProcessBuilder("df", "-k", dir).start();
// assert token count of the data row before letting DF parse it

try {
  df.getMount();
} catch (IOException e) { // 'Could not parse line:'
  // exclude this mount from data dirs; use a standard local filesystem volume
}

When it happens

Trigger: DF.parseOutput() over 'df -k <dir>' output whose data row lacks a trailing field: pseudo-filesystems or FUSE mounts without a mount token, df variants that drop the '%used' column, or the wrapped-device fallback line still missing its tail.

Common situations: Data dirs on FUSE (s3fs, goofys, sshfs) or exotic mounts whose df rows are malformed; minimal Busy df variants; the same wrapped-long-device case as error 517 landing here when the third line exists but is short.

Related errors

• Unexpected empty line
• Expecting additional output after line: ${line}
• Fewer lines of output than expected
• Truncate is not supported by BaiduBosFileSystem
• value cannot be blank

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/26f0f380d633aeff. Report an issue: GitHub.